[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:10 dist-test-slave-04bb krb5kdc[22546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:10 dist-test-slave-04bb krb5kdc[22546](info): set up 2 sockets
Aug 08 04:45:10 dist-test-slave-04bb krb5kdc[22546](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 04:45:11.644872 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.008s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:11 dist-test-slave-04bb krb5kdc[22546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:11.772661 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41365
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:11.898479 22562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:11.898792 22562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:11.898875 22562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:11.902946 22562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:11.903038 22562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:11.903076 22562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:11.903121 22562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:11.903158 22562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:11.908315 22562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:11.909373 22562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:11.910403 22562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:11.919509 22569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:11.919531 22562 server_base.cc:1034] running on GCE node
W20260808 04:45:11.919665 22568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:11.919569 22573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:11.920406 22562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:11.921525 22562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:11.922683 22562 hybrid_clock.cc:648] HybridClock initialized: now 1786164311922662 us; error 76 us; skew 500 ppm
Aug 08 04:45:11 dist-test-slave-04bb krb5kdc[22546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164311, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:11.925760 22562 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:11.926625 22562 webserver.cc:460] Webserver started at http://127.22.2.190:36887/ using document root <none> and password file <none>
I20260808 04:45:11.927181 22562 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:11.927242 22562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:11.927469 22562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:11.929646 22562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "ad8f70dcfbb44b43a4a6a9b841560c16"
format_stamp: "Formatted at 2026-08-08 04:45:11 on dist-test-slave-04bb"
server_key: "d3645255f1f135f0eb6a6deb2d1fe3cb"
server_key_iv: "1832337894a3a137df663bc3bb55d9e7"
server_key_version: "encryptionkey@0"
I20260808 04:45:11.930218 22562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "ad8f70dcfbb44b43a4a6a9b841560c16"
format_stamp: "Formatted at 2026-08-08 04:45:11 on dist-test-slave-04bb"
server_key: "d3645255f1f135f0eb6a6deb2d1fe3cb"
server_key_iv: "1832337894a3a137df663bc3bb55d9e7"
server_key_version: "encryptionkey@0"
I20260808 04:45:11.933949 22562 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:45:11.936427 22581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:11.937683 22562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:11.937839 22562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "ad8f70dcfbb44b43a4a6a9b841560c16"
format_stamp: "Formatted at 2026-08-08 04:45:11 on dist-test-slave-04bb"
server_key: "d3645255f1f135f0eb6a6deb2d1fe3cb"
server_key_iv: "1832337894a3a137df663bc3bb55d9e7"
server_key_version: "encryptionkey@0"
I20260808 04:45:11.937954 22562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:11.976934 22562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:11.980225 22562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:11.980572 22562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:11.991163 22562 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41365
I20260808 04:45:11.991189 22659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41365 every 8 connection(s)
I20260808 04:45:11.992122 22562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:11.995369 22660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:11.999651 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22562
I20260808 04:45:11.999801 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:12.001191 22538 external_mini_cluster.cc:1428] Setting key f94e787fdbdb1fdac14047c10735c9e1
I20260808 04:45:12.002404 22660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16: Bootstrap starting.
I20260808 04:45:12.006466 22660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:12.007810 22660 log.cc:824] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:12.009760 22660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16: No bootstrap required, opened a new log
I20260808 04:45:12.014879 22660 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8f70dcfbb44b43a4a6a9b841560c16" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41365 } }
I20260808 04:45:12.015118 22660 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:12.015200 22660 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8f70dcfbb44b43a4a6a9b841560c16, State: Initialized, Role: FOLLOWER
I20260808 04:45:12.015645 22660 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8f70dcfbb44b43a4a6a9b841560c16" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41365 } }
I20260808 04:45:12.015808 22660 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:12.015877 22660 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:12.015990 22660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:12.017470 22660 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8f70dcfbb44b43a4a6a9b841560c16" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41365 } }
I20260808 04:45:12.017860 22660 leader_election.cc:302] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad8f70dcfbb44b43a4a6a9b841560c16; no voters:
I20260808 04:45:12.018306 22660 leader_election.cc:288] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:12.018410 22669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:12.018662 22669 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [term 1 LEADER]: Becoming Leader. State: Replica: ad8f70dcfbb44b43a4a6a9b841560c16, State: Running, Role: LEADER
I20260808 04:45:12.019006 22669 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8f70dcfbb44b43a4a6a9b841560c16" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41365 } }
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:12.019726 22660 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:12.020234 22669 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad8f70dcfbb44b43a4a6a9b841560c16. Latest consensus state: current_term: 1 leader_uuid: "ad8f70dcfbb44b43a4a6a9b841560c16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8f70dcfbb44b43a4a6a9b841560c16" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41365 } } }
I20260808 04:45:12.020367 22669 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:12.021027 22672 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad8f70dcfbb44b43a4a6a9b841560c16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8f70dcfbb44b43a4a6a9b841560c16" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41365 } } }
I20260808 04:45:12.021142 22672 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:12.022169 22677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:12.025949 22677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:12.029450 22664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.004500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34106 (local address 127.22.2.190:41365)
0808 04:45:12.005014 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:12.005030 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:12.006469 (+ 1439us) server_negotiation.cc:407] Connection header received
0808 04:45:12.009727 (+ 3258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:12.009743 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:12.010104 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:12.010624 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:12.012897 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.013876 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.015061 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.015734 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.015913 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.021102 (+ 5189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:12.021110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:12.021127 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:12.021185 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:12.024673 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.025330 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.025337 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.025344 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.025463 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.025860 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.025864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.025865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.026334 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:12.026631 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:12.028181 (+ 1550us) server_negotiation.cc:299] Negotiation successful
0808 04:45:12.028495 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":154,"threads_started":1}
I20260808 04:45:12.032510 22677 catalog_manager.cc:1347] Generated new cluster ID: 24248fe14bc74a66a67eb70a8379183a
I20260808 04:45:12.032603 22677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:12.066025 22677 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:12.067075 22677 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:12.080569 22677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad8f70dcfbb44b43a4a6a9b841560c16: Generated new TSK 0
I20260808 04:45:12.081375 22677 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:12.125624 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41365
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:12.263168 22702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:12.263419 22702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:12.263461 22702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:12.267307 22702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:12.267361 22702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:12.267436 22702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:12.273068 22702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41365
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:12.274114 22702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:12.275159 22702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:12.284839 22709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:12.285007 22702 server_base.cc:1034] running on GCE node
W20260808 04:45:12.284827 22713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:12.284826 22715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:12.285490 22702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:12.286203 22702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:12.287396 22702 hybrid_clock.cc:648] HybridClock initialized: now 1786164312287384 us; error 31 us; skew 500 ppm
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:12.290288 22702 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:12.291210 22702 webserver.cc:460] Webserver started at http://127.22.2.129:42909/ using document root <none> and password file <none>
I20260808 04:45:12.291783 22702 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:12.291893 22702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:12.292133 22702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:12.294304 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "c0abec81316242b6aed8bd4415cac68d"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "755b76f9af1469224aae69f9cc9d29c9"
server_key_iv: "c6d10f597d0787e38c9f6d738287cce1"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.294909 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "c0abec81316242b6aed8bd4415cac68d"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "755b76f9af1469224aae69f9cc9d29c9"
server_key_iv: "c6d10f597d0787e38c9f6d738287cce1"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.298629 22702 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 04:45:12.301146 22724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.302337 22702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:12.302480 22702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "c0abec81316242b6aed8bd4415cac68d"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "755b76f9af1469224aae69f9cc9d29c9"
server_key_iv: "c6d10f597d0787e38c9f6d738287cce1"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.302596 22702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:12.312471 22702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:12.315826 22702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:12.316030 22702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:12.316646 22702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:12.317485 22702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:12.317559 22702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.317631 22702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:12.317675 22702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.327656 22702 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:32917
I20260808 04:45:12.327682 22903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:32917 every 8 connection(s)
I20260808 04:45:12.328603 22702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:12.333318 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22702
I20260808 04:45:12.333405 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:12.333669 22538 external_mini_cluster.cc:1428] Setting key 5f715cd3853e4308608443d3e6b703e3
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:12.343587 22664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.330552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:59079 (local address 127.22.2.190:41365)
0808 04:45:12.330694 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:12.330697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:12.331574 (+ 877us) server_negotiation.cc:407] Connection header received
0808 04:45:12.332978 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:12.332982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:12.333044 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:12.333141 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:12.334928 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.335453 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.336665 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.336928 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.337070 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.339764 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:12.339767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:12.339769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:12.339794 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:12.341359 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.341946 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.341951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.341953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.342069 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.342428 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.342433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.342434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.342622 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:12.342732 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:12.343267 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 04:45:12.343411 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:45:12.344455 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.330913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41365 (local address 127.22.2.129:59079)
0808 04:45:12.331408 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:12.331449 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:12.332755 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:12.333337 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:12.333343 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:12.333830 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:12.334321 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:12.334336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.335587 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.335591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:12.336490 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:12.336499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.337171 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.337178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:12.337419 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.338070 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:12.338087 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:12.339566 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:12.341489 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:12.341495 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:12.341511 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:12.341834 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:12.342176 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:12.342180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:12.342182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:12.342331 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:12.342733 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:12.342741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:12.343022 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 04:45:12.343212 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 04:45:12.343536 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":137,"threads_started":1}
I20260808 04:45:12.345805 22904 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41365
I20260808 04:45:12.346032 22904 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:12.346607 22904 heartbeater.cc:511] Master 127.22.2.190:41365 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 04:45:12.348357 22603 ts_manager.cc:194] Registered new tserver with Master: c0abec81316242b6aed8bd4415cac68d (127.22.2.129:32917)
I20260808 04:45:12.349731 22603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:59079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:12.365180 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41365
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:12.483600 22912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:12.483846 22912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:12.483891 22912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:12.487941 22912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:12.487994 22912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:12.488085 22912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:12.493358 22912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41365
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:12.494374 22912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:12.495471 22912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:12.505136 22922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:12.505134 22918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:12.505146 22920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:12.505445 22912 server_base.cc:1034] running on GCE node
I20260808 04:45:12.505985 22912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:12.506711 22912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:12.507910 22912 hybrid_clock.cc:648] HybridClock initialized: now 1786164312507873 us; error 50 us; skew 500 ppm
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:12.510877 22912 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:12.511826 22912 webserver.cc:460] Webserver started at http://127.22.2.130:40037/ using document root <none> and password file <none>
I20260808 04:45:12.512406 22912 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:12.512467 22912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:12.512715 22912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:12.514731 22912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "7bf181004dac4e2cba9e15d0f0098e86"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "1c890cc6a714d7381d9b9318bf289c99"
server_key_iv: "3189588c8cf57a9af6144a3dae229721"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.515261 22912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "7bf181004dac4e2cba9e15d0f0098e86"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "1c890cc6a714d7381d9b9318bf289c99"
server_key_iv: "3189588c8cf57a9af6144a3dae229721"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.519084 22912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:45:12.521822 22932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.523244 22912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 04:45:12.523350 22912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "7bf181004dac4e2cba9e15d0f0098e86"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "1c890cc6a714d7381d9b9318bf289c99"
server_key_iv: "3189588c8cf57a9af6144a3dae229721"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.523437 22912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:12.552898 22912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:12.556326 22912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:12.556488 22912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:12.557066 22912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:12.557917 22912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:12.557965 22912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.558007 22912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:12.558022 22912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.567312 22912 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:43169
I20260808 04:45:12.567346 23097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:43169 every 8 connection(s)
I20260808 04:45:12.568240 22912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:12.572247 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22912
I20260808 04:45:12.572348 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:12.572647 22538 external_mini_cluster.cc:1428] Setting key 36a326ec8d3efd1237b1b9329502b6b3
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:45:12.584386 22664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.570187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:34957 (local address 127.22.2.190:41365)
0808 04:45:12.570340 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:12.570345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:12.571260 (+ 915us) server_negotiation.cc:407] Connection header received
0808 04:45:12.572836 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:12.572840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:12.572902 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:12.572984 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:12.574550 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.575161 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.576929 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.577356 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.577509 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.580680 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:12.580683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:12.580685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:12.580710 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:12.582274 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.582788 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.582797 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.582798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.582875 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.583235 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.583239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.583241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.583422 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:12.583533 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:12.584089 (+ 556us) server_negotiation.cc:299] Negotiation successful
0808 04:45:12.584249 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:45:12.585161 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.570623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41365 (local address 127.22.2.130:34957)
0808 04:45:12.571099 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:12.571142 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:12.572565 (+ 1423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:12.573210 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:12.573218 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:12.573703 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:12.574354 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:12.574376 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.575317 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.575324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:12.576790 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:12.576801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.577533 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.577537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:12.577679 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.578314 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:12.578330 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:12.580460 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:12.582398 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:12.582404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:12.582420 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:12.582675 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:12.582977 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:12.582980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:12.582983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:12.583125 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:12.583532 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:12.583539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:12.583786 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 04:45:12.583976 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 04:45:12.584257 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":118,"threads_started":1}
I20260808 04:45:12.586459 23099 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41365
I20260808 04:45:12.586755 23099 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:45:12.587365 23099 heartbeater.cc:511] Master 127.22.2.190:41365 requested a full tablet report, sending...
I20260808 04:45:12.588696 22603 ts_manager.cc:194] Registered new tserver with Master: 7bf181004dac4e2cba9e15d0f0098e86 (127.22.2.130:43169)
I20260808 04:45:12.589330 22603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:34957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:12.603823 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41365
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:12.724198 23107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:12.724815 23107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:12.724996 23107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:12.731535 23107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:12.731618 23107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:12.731709 23107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:12.741233 23107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41365
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:12.742774 23107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:12.744190 23107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:12.756935 23122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:12.757081 23112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:12.756992 23107 server_base.cc:1034] running on GCE node
W20260808 04:45:12.757279 23113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:12.757747 23107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:12.758498 23107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:12.759663 23107 hybrid_clock.cc:648] HybridClock initialized: now 1786164312759586 us; error 99 us; skew 500 ppm
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:12.762598 23107 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:12.763710 23107 webserver.cc:460] Webserver started at http://127.22.2.131:33499/ using document root <none> and password file <none>
I20260808 04:45:12.764461 23107 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:12.764582 23107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:12.764842 23107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:12.766881 23107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "5537df016c114ab4a7c406d7dbaa756f"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "a29f3c0aa11294a842b64ff094ff2865"
server_key_iv: "16820636b981a66a383298a70fbc9aa0"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.767482 23107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "5537df016c114ab4a7c406d7dbaa756f"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "a29f3c0aa11294a842b64ff094ff2865"
server_key_iv: "16820636b981a66a383298a70fbc9aa0"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.772212 23107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 04:45:12.774717 23139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.776217 23107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:12.776376 23107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "5537df016c114ab4a7c406d7dbaa756f"
format_stamp: "Formatted at 2026-08-08 04:45:12 on dist-test-slave-04bb"
server_key: "a29f3c0aa11294a842b64ff094ff2865"
server_key_iv: "16820636b981a66a383298a70fbc9aa0"
server_key_version: "encryptionkey@0"
I20260808 04:45:12.776549 23107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:12.793324 23107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:12.796463 23107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:12.796702 23107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:12.797320 23107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:12.798171 23107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:12.798241 23107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.798308 23107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:12.798357 23107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:12.808121 23107 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:38091
I20260808 04:45:12.808151 23296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:38091 every 8 connection(s)
I20260808 04:45:12.809091 23107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:12.810906 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23107
I20260808 04:45:12.811039 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:12.811344 22538 external_mini_cluster.cc:1428] Setting key 88b516208b38be82689c65dabed5024f
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:12.823434 22664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.811051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:36839 (local address 127.22.2.190:41365)
0808 04:45:12.811944 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:12.811951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:12.812161 (+ 210us) server_negotiation.cc:407] Connection header received
0808 04:45:12.813644 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:12.813648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:12.813702 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:12.813806 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:12.815123 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.815679 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.816864 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.817076 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.817160 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.819740 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:12.819744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:12.819746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:12.819779 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:12.821228 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.821738 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.821742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.821744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.821833 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.822284 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.822289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.822291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.822485 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:12.822595 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:12.823083 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 04:45:12.823210 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781}
I20260808 04:45:12.824186 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.811502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41365 (local address 127.22.2.131:36839)
0808 04:45:12.811997 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:12.812050 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:12.813434 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:12.813997 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:12.814006 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:12.814456 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:12.814973 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:12.814987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.815787 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.815789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:12.816732 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:12.816742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.817204 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.817207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:12.817413 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.817939 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:12.817956 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:12.819541 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:12.821344 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:12.821349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:12.821365 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:12.821623 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:12.821930 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:12.821933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:12.821935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:12.822085 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:12.822594 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:12.822601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:12.822846 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 04:45:12.823038 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 04:45:12.823366 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":144,"threads_started":1}
I20260808 04:45:12.825438 23297 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41365
I20260808 04:45:12.825696 23297 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:12.826193 23297 heartbeater.cc:511] Master 127.22.2.190:41365 requested a full tablet report, sending...
I20260808 04:45:12.827323 22603 ts_manager.cc:194] Registered new tserver with Master: 5537df016c114ab4a7c406d7dbaa756f (127.22.2.131:38091)
I20260808 04:45:12.827988 22603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:36839
I20260808 04:45:12.837822 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:12.865378 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.856086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55852 (local address 127.22.2.129:32917)
0808 04:45:12.856421 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:12.856428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:12.856453 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:45:12.856604 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:12.856610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:12.856838 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:12.856966 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:12.857637 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.858385 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.859190 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.859553 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.859634 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.861989 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:12.861997 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:12.862010 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:12.862035 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:12.863677 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.864056 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.864061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.864062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.864141 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.864377 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.864380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.864381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.864700 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:12.864867 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:12.865019 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 04:45:12.865179 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":123,"threads_started":1}
W20260808 04:45:12.866387 22767 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55852
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[22546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:12.894596 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:12.885795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55866 (local address 127.22.2.129:32917)
0808 04:45:12.885947 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:12.885952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:12.886108 (+ 156us) server_negotiation.cc:407] Connection header received
0808 04:45:12.886274 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:12.886277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:12.886329 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:12.886403 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:12.887262 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.887756 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.888704 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:12.888923 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:12.889001 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:12.891598 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:12.891602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:12.891604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:12.891627 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:12.893177 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.893588 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.893591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.893592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.893672 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:12.893924 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:12.893927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:12.893928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:12.894087 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:12.894246 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:12.894375 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 04:45:12.894473 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 04:45:12.895763 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22702
I20260808 04:45:12.905283 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22912
I20260808 04:45:12.920107 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23107
I20260808 04:45:12.927963 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22562
2026-08-08T04:45:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2554 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[23319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[23319](info): set up 2 sockets
Aug 08 04:45:12 dist-test-slave-04bb krb5kdc[23319](info): commencing operation
krb5kdc: starting...
W20260808 04:45:14.177744 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:14 dist-test-slave-04bb krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:14.307188 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34203
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:34203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:14.421885 23335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:14.422135 23335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:14.422174 23335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:14.426367 23335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:14.426421 23335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:14.426441 23335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:14.426458 23335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:14.426499 23335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:14.431710 23335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:34203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:14.432771 23335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:14.433818 23335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:14.442435 23346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:14.442437 23344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:14.442497 23342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:14.442621 23335 server_base.cc:1034] running on GCE node
I20260808 04:45:14.443286 23335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:14.444383 23335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:14.445559 23335 hybrid_clock.cc:648] HybridClock initialized: now 1786164314445545 us; error 30 us; skew 500 ppm
Aug 08 04:45:14 dist-test-slave-04bb krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164314, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:14.448410 23335 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:14.449332 23335 webserver.cc:460] Webserver started at http://127.22.2.190:37131/ using document root <none> and password file <none>
I20260808 04:45:14.449900 23335 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:14.449999 23335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:14.450246 23335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:14.452306 23335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "f1773b052855458166759dda12282a8c"
server_key_iv: "602eed08b7da37c4ea951e1bf53b006b"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.452989 23335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "f1773b052855458166759dda12282a8c"
server_key_iv: "602eed08b7da37c4ea951e1bf53b006b"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.456703 23335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:45:14.459197 23353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:14.460616 23335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:14.460754 23335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "f1773b052855458166759dda12282a8c"
server_key_iv: "602eed08b7da37c4ea951e1bf53b006b"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.460860 23335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:14.480755 23335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:14.483774 23335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:14.483974 23335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:14.491317 23335 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:34203
I20260808 04:45:14.491341 23425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:34203 every 8 connection(s)
I20260808 04:45:14.492270 23335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:14.493376 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23335
I20260808 04:45:14.493465 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:14.493731 22538 external_mini_cluster.cc:1428] Setting key db5d112f027f6fab4c5fb7f0380200a6
I20260808 04:45:14.495498 23426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:14.502465 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6: Bootstrap starting.
Aug 08 04:45:14 dist-test-slave-04bb krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:14.505247 23426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:14.506141 23426 log.cc:824] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:14.508024 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6: No bootstrap required, opened a new log
I20260808 04:45:14.509195 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:14.495160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33446 (local address 127.22.2.190:34203)
0808 04:45:14.495601 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:14.495621 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:14.495665 (+ 44us) server_negotiation.cc:407] Connection header received
0808 04:45:14.496488 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:14.496513 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:14.496954 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:14.497317 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:14.497980 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.499508 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.500398 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.501142 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.501326 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:14.504225 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:14.504233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:14.504248 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:14.504279 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:14.506305 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:14.506747 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:14.506754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:14.506765 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:14.506892 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:14.507190 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:14.507194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:14.507196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:14.507583 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:14.507797 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:14.508125 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 04:45:14.508448 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":129,"threads_started":1}
I20260808 04:45:14.511718 23426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34203 } }
I20260808 04:45:14.511968 23426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:14.512045 23426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c2fabe88d4da3a51600b7bd3ccab6, State: Initialized, Role: FOLLOWER
I20260808 04:45:14.512483 23426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34203 } }
I20260808 04:45:14.512683 23426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:14.512775 23426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:14.512873 23426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:14.513731 23426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34203 } }
I20260808 04:45:14.514065 23426 leader_election.cc:302] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a2c2fabe88d4da3a51600b7bd3ccab6; no voters:
I20260808 04:45:14.514403 23426 leader_election.cc:288] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:14.514541 23436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:14.514818 23436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [term 1 LEADER]: Becoming Leader. State: Replica: 4a2c2fabe88d4da3a51600b7bd3ccab6, State: Running, Role: LEADER
I20260808 04:45:14.515129 23436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34203 } }
I20260808 04:45:14.515656 23426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:14.516865 23439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a2c2fabe88d4da3a51600b7bd3ccab6. Latest consensus state: current_term: 1 leader_uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34203 } } }
I20260808 04:45:14.516999 23439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:14.517328 23437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2c2fabe88d4da3a51600b7bd3ccab6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34203 } } }
I20260808 04:45:14.517449 23437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6 [sys.catalog]: This master's current role is: LEADER
W20260808 04:45:14.520426 23458 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:45:14.520494 23458 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:45:14.520628 23450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:14.521597 23450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:14.526924 23450 catalog_manager.cc:1347] Generated new cluster ID: 2d19164612bb4ee69ad6d5f1faa2fafa
I20260808 04:45:14.526988 23450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:14.543498 23450 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:14.544448 23450 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:14.552281 23450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a2c2fabe88d4da3a51600b7bd3ccab6: Generated new TSK 0
I20260808 04:45:14.552966 23450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:14.591945 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34203
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:14.710557 23468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:14.710806 23468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:14.710850 23468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:14.714692 23468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:14.714736 23468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:14.714792 23468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:14.720247 23468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:14.721232 23468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:14.722213 23468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:14.731773 23478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:14.731768 23476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:14.731841 23475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:14.732020 23468 server_base.cc:1034] running on GCE node
I20260808 04:45:14.732489 23468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:14.733168 23468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:14.734340 23468 hybrid_clock.cc:648] HybridClock initialized: now 1786164314734326 us; error 25 us; skew 500 ppm
Aug 08 04:45:14 dist-test-slave-04bb krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164314, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:14.737386 23468 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:14.738209 23468 webserver.cc:460] Webserver started at http://127.22.2.129:41265/ using document root <none> and password file <none>
I20260808 04:45:14.738736 23468 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:14.738794 23468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:14.738962 23468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:14.741000 23468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "64b801e7c43847408ce5c57d8035dc56"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "4c5f9addd79e1ccb5230296d41ee0537"
server_key_iv: "745fc6b47338c63d955c5d447f6d8a5e"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.741520 23468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "64b801e7c43847408ce5c57d8035dc56"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "4c5f9addd79e1ccb5230296d41ee0537"
server_key_iv: "745fc6b47338c63d955c5d447f6d8a5e"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.745136 23468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:45:14.747529 23492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:14.748854 23468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:14.748948 23468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "64b801e7c43847408ce5c57d8035dc56"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "4c5f9addd79e1ccb5230296d41ee0537"
server_key_iv: "745fc6b47338c63d955c5d447f6d8a5e"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.749029 23468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:14.762172 23468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:14.765455 23468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:14.765605 23468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:14.766127 23468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:14.766899 23468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:14.766956 23468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:14.766996 23468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:14.767017 23468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:14.775851 23468 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:46415
I20260808 04:45:14.775876 23660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:46415 every 8 connection(s)
I20260808 04:45:14.776818 23468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:14.778658 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23468
I20260808 04:45:14.778770 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:14.779083 22538 external_mini_cluster.cc:1428] Setting key 6675b0f7fdb436e1781a03476bc42f1d
Aug 08 04:45:14 dist-test-slave-04bb krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164314, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:45:14.791739 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:14.778715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:58733 (local address 127.22.2.190:34203)
0808 04:45:14.778877 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:14.778883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:14.779917 (+ 1034us) server_negotiation.cc:407] Connection header received
0808 04:45:14.781076 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:14.781080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:14.781134 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:14.781203 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:14.782426 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.783009 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.784814 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.785036 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.785118 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:14.788013 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:14.788017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:14.788021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:14.788047 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:14.789532 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:14.790120 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:14.790123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:14.790125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:14.790200 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:14.790542 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:14.790546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:14.790548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:14.790772 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:14.790881 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:14.791378 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 04:45:14.791510 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 04:45:14.792678 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:14.779166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34203 (local address 127.22.2.129:58733)
0808 04:45:14.779729 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:14.779810 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:14.780858 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:14.781370 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:14.781377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:14.781732 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:14.782228 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:14.782241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.783145 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.783152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:14.784685 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:14.784701 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.785201 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.785208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:14.785476 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:14.786138 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:14.786162 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:14.787852 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:14.789666 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:14.789672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:14.789687 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:14.789998 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:14.790293 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:14.790297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:14.790299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:14.790442 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:14.790882 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:14.790889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:14.791137 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 04:45:14.791336 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 04:45:14.791653 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":138,"threads_started":1}
I20260808 04:45:14.793908 23661 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34203
I20260808 04:45:14.794149 23661 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:14.794668 23661 heartbeater.cc:511] Master 127.22.2.190:34203 requested a full tablet report, sending...
I20260808 04:45:14.796192 23374 ts_manager.cc:194] Registered new tserver with Master: 64b801e7c43847408ce5c57d8035dc56 (127.22.2.129:46415)
I20260808 04:45:14.797381 23374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:58733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:14.808117 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34203
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:14.922950 23671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:14.923190 23671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:14.923233 23671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:14.927208 23671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:14.927263 23671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:14.927318 23671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:14.932281 23671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:14.933274 23671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:14.934298 23671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:14.944049 23678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:14.944144 23679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:14.944074 23684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:14.944386 23671 server_base.cc:1034] running on GCE node
I20260808 04:45:14.944826 23671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:14.945484 23671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:14.946684 23671 hybrid_clock.cc:648] HybridClock initialized: now 1786164314946668 us; error 29 us; skew 500 ppm
Aug 08 04:45:14 dist-test-slave-04bb krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164314, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:14.949666 23671 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:14.950557 23671 webserver.cc:460] Webserver started at http://127.22.2.130:33497/ using document root <none> and password file <none>
I20260808 04:45:14.951166 23671 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:14.951277 23671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:14.951555 23671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:14.953759 23671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "b898fac4b2304c3c943c8d248f281913"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "fb4230fe7ceff808a3cbcf7537956faf"
server_key_iv: "ce826a6dafb8cf1d8b75768928fb40ce"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.954344 23671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "b898fac4b2304c3c943c8d248f281913"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "fb4230fe7ceff808a3cbcf7537956faf"
server_key_iv: "ce826a6dafb8cf1d8b75768928fb40ce"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.958067 23671 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:45:14.960670 23695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:14.961889 23671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:45:14.962045 23671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "b898fac4b2304c3c943c8d248f281913"
format_stamp: "Formatted at 2026-08-08 04:45:14 on dist-test-slave-04bb"
server_key: "fb4230fe7ceff808a3cbcf7537956faf"
server_key_iv: "ce826a6dafb8cf1d8b75768928fb40ce"
server_key_version: "encryptionkey@0"
I20260808 04:45:14.962154 23671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:14.976202 23671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:14.979501 23671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:14.979712 23671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:14.980337 23671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:14.981230 23671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:14.981305 23671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:14.981369 23671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:14.981417 23671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:14.990306 23671 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:43787
I20260808 04:45:14.990362 23860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:43787 every 8 connection(s)
I20260808 04:45:14.991261 23671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:14.994825 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23671
I20260808 04:45:14.994940 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:14.995237 22538 external_mini_cluster.cc:1428] Setting key d1681ad456c5d22289e1e55f1dbf4585
Aug 08 04:45:14 dist-test-slave-04bb krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164314, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:45:15.005887 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:14.993221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53959 (local address 127.22.2.190:34203)
0808 04:45:14.993363 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:14.993367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:14.994222 (+ 855us) server_negotiation.cc:407] Connection header received
0808 04:45:14.995430 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:14.995434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:14.995482 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:14.995556 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:14.997052 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.997829 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.999153 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.999377 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.999470 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.002226 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:15.002229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:15.002232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:15.002255 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:15.003762 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.004279 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.004289 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.004291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.004365 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.004774 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.004778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.004780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.004968 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:15.005096 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.005585 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.005714 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:15.006656 23864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:14.993634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34203 (local address 127.22.2.130:53959)
0808 04:45:14.994079 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:14.994121 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:14.995176 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:14.995766 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:14.995774 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:14.996212 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:14.996907 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:14.996923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.997962 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.997964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:14.999019 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:14.999033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:14.999543 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:14.999549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:14.999764 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.000572 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:15.000602 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:15.002043 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:15.003883 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:15.003889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:15.003903 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:15.004166 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:15.004459 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:15.004464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:15.004466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:15.004674 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:15.005080 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:15.005088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:15.005345 (+ 257us) client_negotiation.cc:769] Sending connection context
0808 04:45:15.005524 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 04:45:15.005835 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
I20260808 04:45:15.007843 23862 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34203
I20260808 04:45:15.008116 23862 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:15.008759 23862 heartbeater.cc:511] Master 127.22.2.190:34203 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:45:15.009856 23374 ts_manager.cc:194] Registered new tserver with Master: b898fac4b2304c3c943c8d248f281913 (127.22.2.130:43787)
I20260808 04:45:15.010464 23374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:53959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:15.025578 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34203
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:15.140925 23867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:15.141161 23867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:15.141225 23867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:15.145249 23867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:15.145299 23867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:15.145393 23867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:15.150537 23867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:15.151539 23867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:15.152700 23867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:15.162416 23878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:15.162389 23876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:15.162381 23875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:15.162815 23867 server_base.cc:1034] running on GCE node
I20260808 04:45:15.163218 23867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:15.163771 23867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:15.164933 23867 hybrid_clock.cc:648] HybridClock initialized: now 1786164315164901 us; error 44 us; skew 500 ppm
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164315, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:15.167778 23867 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:15.168736 23867 webserver.cc:460] Webserver started at http://127.22.2.131:37175/ using document root <none> and password file <none>
I20260808 04:45:15.169298 23867 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:15.169390 23867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:15.169631 23867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:15.171614 23867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "0a07ab2e431a4115a513dfd0afbfec14"
format_stamp: "Formatted at 2026-08-08 04:45:15 on dist-test-slave-04bb"
server_key: "f40e15216f6de027d0dd32e02fcd7f28"
server_key_iv: "c3aff965a701a41ad957bf9e7c6a5dff"
server_key_version: "encryptionkey@0"
I20260808 04:45:15.172184 23867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "0a07ab2e431a4115a513dfd0afbfec14"
format_stamp: "Formatted at 2026-08-08 04:45:15 on dist-test-slave-04bb"
server_key: "f40e15216f6de027d0dd32e02fcd7f28"
server_key_iv: "c3aff965a701a41ad957bf9e7c6a5dff"
server_key_version: "encryptionkey@0"
I20260808 04:45:15.175871 23867 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 04:45:15.178426 23887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:15.179734 23867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 04:45:15.179911 23867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "0a07ab2e431a4115a513dfd0afbfec14"
format_stamp: "Formatted at 2026-08-08 04:45:15 on dist-test-slave-04bb"
server_key: "f40e15216f6de027d0dd32e02fcd7f28"
server_key_iv: "c3aff965a701a41ad957bf9e7c6a5dff"
server_key_version: "encryptionkey@0"
I20260808 04:45:15.180038 23867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:15.220727 23867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:15.224162 23867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:15.224387 23867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:15.225152 23867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:15.226071 23867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:15.226161 23867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:15.226241 23867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:15.226289 23867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:15.235746 23867 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:38959
I20260808 04:45:15.235770 24062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:38959 every 8 connection(s)
I20260808 04:45:15.236717 23867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:15.242671 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23867
I20260808 04:45:15.242795 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:15.243081 22538 external_mini_cluster.cc:1428] Setting key de243f0b4547ca0dfaf718ca05e75502
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164315, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:15.250788 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.238767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:54615 (local address 127.22.2.190:34203)
0808 04:45:15.238925 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.238929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.239991 (+ 1062us) server_negotiation.cc:407] Connection header received
0808 04:45:15.241026 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.241029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.241092 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.241201 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:15.242399 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.243191 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.244696 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.244928 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.245047 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.247567 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:15.247570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:15.247572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:15.247596 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:15.248998 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.249470 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.249473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.249474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.249554 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.249856 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.249859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.249860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.250030 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:15.250106 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.250560 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.250672 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:45:15.251685 24065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.239157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34203 (local address 127.22.2.131:54615)
0808 04:45:15.239820 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:15.239876 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:15.240809 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:15.241384 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:15.241390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:15.241740 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:15.242249 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.242263 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.243335 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.243338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.244511 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.244575 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.245090 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.245094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.245315 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.245901 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:15.245917 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:15.247411 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:15.249111 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:15.249115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:15.249127 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:15.249376 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:15.249635 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:15.249638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:15.249639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:15.249765 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:15.250116 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:15.250121 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:15.250342 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 04:45:15.250498 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 04:45:15.250737 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":120,"threads_started":1}
I20260808 04:45:15.252951 24063 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34203
I20260808 04:45:15.253309 24063 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:15.253818 24063 heartbeater.cc:511] Master 127.22.2.190:34203 requested a full tablet report, sending...
I20260808 04:45:15.254849 23374 ts_manager.cc:194] Registered new tserver with Master: 0a07ab2e431a4115a513dfd0afbfec14 (127.22.2.131:38959)
I20260808 04:45:15.255477 23374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:54615
I20260808 04:45:15.257089 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:45:15.268049 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.260603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.22.2.190:34203)
0808 04:45:15.260738 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.260742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.260911 (+ 169us) server_negotiation.cc:407] Connection header received
0808 04:45:15.260989 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.260991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.261029 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.261092 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:15.261739 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.262212 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.262974 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.263171 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.263243 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.264731 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:15.264734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:15.264736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:15.264757 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:15.266293 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.266763 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.266766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.266767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.266848 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.267117 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.267121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.267122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.267357 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:15.267475 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.267796 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.267925 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:45:15.272890 23374 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:15.275780 23374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:15.291005 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.284422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.22.2.129:46415)
0808 04:45:15.285771 (+ 1349us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.285787 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.285811 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:45:15.286015 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.286021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.286259 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.286415 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:15.286887 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.287906 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.289328 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.290167 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.290287 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.290436 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.290739 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.290838 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":62,"threads_started":1}
I20260808 04:45:15.291025 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.284417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38959 (local address 127.0.0.1:36668)
0808 04:45:15.285125 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:15.285142 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:15.285237 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:15.285869 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:15.285876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:15.285901 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:15.286054 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.286063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.287667 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.287670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.289274 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.289286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.290303 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.290307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.290468 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.290574 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 04:45:15.290689 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 04:45:15.290845 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":223,"thread_start_us":138,"threads_started":1}
I20260808 04:45:15.291025 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.284312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46415 (local address 127.0.0.1:56914)
0808 04:45:15.284893 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:15.284944 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:15.285070 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:15.286617 (+ 1547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:15.286620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:15.286642 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:15.286749 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.286756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.288047 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.288052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.289197 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.289207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.290303 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.290307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.290471 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.290575 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 04:45:15.290689 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 04:45:15.290845 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":99,"thread_start_us":92,"threads_started":1}
I20260808 04:45:15.291633 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.284830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.22.2.131:38959)
0808 04:45:15.285297 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.285315 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.285339 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:45:15.285420 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.285426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.285673 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.286204 (+ 531us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:15.286330 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.287467 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.289383 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.290167 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.290287 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.290436 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.291213 (+ 777us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.291325 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":143,"threads_started":1}
I20260808 04:45:15.292075 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.285077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:43787 (local address 127.0.0.1:58844)
0808 04:45:15.285446 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:15.285458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:15.285531 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:15.286419 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:15.286422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:15.286441 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:15.286525 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.286531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.287590 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.287595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.289274 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.289286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.291632 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.291636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.291747 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.291777 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 04:45:15.291869 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 04:45:15.291948 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":125,"thread_start_us":95,"threads_started":1}
I20260808 04:45:15.292589 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.285526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.22.2.130:43787)
0808 04:45:15.285959 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.285972 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.285995 (+ 23us) server_negotiation.cc:407] Connection header received
0808 04:45:15.286064 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.286068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.286259 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.286415 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:15.286647 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.287458 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.290510 (+ 3052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.291304 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.291415 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.291561 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.292282 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.292411 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":125,"threads_started":1}
I20260808 04:45:15.294718 23954 tablet_service.cc:1467] Processing CreateTablet for tablet 7af05302aeda4accb876f561fb546370 (DEFAULT_TABLE table=test-table [id=332560aee6da49a2a598c8441d6dfa9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:15.294767 23565 tablet_service.cc:1467] Processing CreateTablet for tablet 7af05302aeda4accb876f561fb546370 (DEFAULT_TABLE table=test-table [id=332560aee6da49a2a598c8441d6dfa9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:15.295212 23767 tablet_service.cc:1467] Processing CreateTablet for tablet 7af05302aeda4accb876f561fb546370 (DEFAULT_TABLE table=test-table [id=332560aee6da49a2a598c8441d6dfa9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:15.295853 23954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af05302aeda4accb876f561fb546370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:15.295886 23565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af05302aeda4accb876f561fb546370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:15.296114 23767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af05302aeda4accb876f561fb546370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:15.301999 24086 tablet_bootstrap.cc:492] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14: Bootstrap starting.
I20260808 04:45:15.303608 24087 tablet_bootstrap.cc:492] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913: Bootstrap starting.
I20260808 04:45:15.304406 24086 tablet_bootstrap.cc:654] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:15.304795 24088 tablet_bootstrap.cc:492] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56: Bootstrap starting.
I20260808 04:45:15.305768 24086 log.cc:824] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:15.306420 24087 tablet_bootstrap.cc:654] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:15.306722 24088 tablet_bootstrap.cc:654] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:15.307197 24087 log.cc:824] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:15.307420 24088 log.cc:824] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:15.308835 24086 tablet_bootstrap.cc:492] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14: No bootstrap required, opened a new log
I20260808 04:45:15.309042 24086 ts_tablet_manager.cc:1397] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 04:45:15.309275 24088 tablet_bootstrap.cc:492] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56: No bootstrap required, opened a new log
I20260808 04:45:15.309275 24087 tablet_bootstrap.cc:492] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913: No bootstrap required, opened a new log
I20260808 04:45:15.309460 24087 ts_tablet_manager.cc:1397] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:45:15.309460 24088 ts_tablet_manager.cc:1397] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:45:15.311918 24086 raft_consensus.cc:348] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.312186 24086 raft_consensus.cc:374] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:15.312253 24086 raft_consensus.cc:729] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a07ab2e431a4115a513dfd0afbfec14, State: Initialized, Role: FOLLOWER
I20260808 04:45:15.312793 24086 consensus_queue.cc:260] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.313405 24087 raft_consensus.cc:348] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.313666 24087 raft_consensus.cc:374] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:15.313735 24087 raft_consensus.cc:729] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b898fac4b2304c3c943c8d248f281913, State: Initialized, Role: FOLLOWER
I20260808 04:45:15.313894 24086 ts_tablet_manager.cc:1428] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 04:45:15.314148 24087 consensus_queue.cc:260] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.314293 24063 heartbeater.cc:503] Master 127.22.2.190:34203 was elected leader, sending a full tablet report...
I20260808 04:45:15.314136 24088 raft_consensus.cc:348] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.314366 24088 raft_consensus.cc:374] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:15.314425 24088 raft_consensus.cc:729] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b801e7c43847408ce5c57d8035dc56, State: Initialized, Role: FOLLOWER
I20260808 04:45:15.314810 23862 heartbeater.cc:503] Master 127.22.2.190:34203 was elected leader, sending a full tablet report...
I20260808 04:45:15.314867 24088 consensus_queue.cc:260] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.315177 24087 ts_tablet_manager.cc:1428] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:45:15.315881 24088 ts_tablet_manager.cc:1428] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:45:15.316210 23661 heartbeater.cc:503] Master 127.22.2.190:34203 was elected leader, sending a full tablet report...
W20260808 04:45:15.487775 24064 tablet.cc:2369] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:45:15.492468 23863 tablet.cc:2369] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:15.507392 24098 raft_consensus.cc:482] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:15.507639 24098 raft_consensus.cc:504] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.508873 24098 leader_election.cc:288] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a07ab2e431a4115a513dfd0afbfec14 (127.22.2.131:38959), b898fac4b2304c3c943c8d248f281913 (127.22.2.130:43787)
I20260808 04:45:15.512993 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.509341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48453 (local address 127.22.2.130:43787)
0808 04:45:15.509506 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.509510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.509699 (+ 189us) server_negotiation.cc:407] Connection header received
0808 04:45:15.509844 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.509847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.509898 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.509965 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:15.510352 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.510856 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.512034 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.512406 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.512531 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.512609 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.512793 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.512865 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 04:45:15.513016 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.509266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:36541 (local address 127.22.2.131:38959)
0808 04:45:15.509415 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.509420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.509642 (+ 222us) server_negotiation.cc:407] Connection header received
0808 04:45:15.509884 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.509887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.509932 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.510028 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:15.510391 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.510861 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.512004 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.512413 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.512579 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.512636 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.512842 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.512914 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:15.513032 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.509222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:43787 (local address 127.22.2.129:48453)
0808 04:45:15.509566 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:15.509578 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:15.509729 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:15.510010 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:15.510016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:15.510034 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:15.510231 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.510238 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.510979 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.510986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.511880 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.511890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.512552 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.512556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.512658 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.512688 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 04:45:15.512751 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 04:45:15.512849 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":117,"spinlock_wait_cycles":1408,"thread_start_us":129,"threads_started":1}
I20260808 04:45:15.513023 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.509166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38959 (local address 127.22.2.129:36541)
0808 04:45:15.509519 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:15.509536 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:15.509727 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:15.510059 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:15.510065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:15.510083 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:15.510242 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.510248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.511002 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.511005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.511875 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:15.511884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.512577 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.512581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:15.512690 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.512708 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 04:45:15.512805 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 04:45:15.512873 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":25,"spinlock_wait_cycles":29312,"thread_start_us":110,"threads_started":1}
I20260808 04:45:15.513881 23795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af05302aeda4accb876f561fb546370" candidate_uuid: "64b801e7c43847408ce5c57d8035dc56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b898fac4b2304c3c943c8d248f281913" is_pre_election: true
I20260808 04:45:15.513881 23985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af05302aeda4accb876f561fb546370" candidate_uuid: "64b801e7c43847408ce5c57d8035dc56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a07ab2e431a4115a513dfd0afbfec14" is_pre_election: true
I20260808 04:45:15.514191 23795 raft_consensus.cc:2454] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b801e7c43847408ce5c57d8035dc56 in term 0.
I20260808 04:45:15.514191 23985 raft_consensus.cc:2454] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b801e7c43847408ce5c57d8035dc56 in term 0.
I20260808 04:45:15.514739 23494 leader_election.cc:302] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a07ab2e431a4115a513dfd0afbfec14, 64b801e7c43847408ce5c57d8035dc56, b898fac4b2304c3c943c8d248f281913; no voters:
I20260808 04:45:15.515043 24098 raft_consensus.cc:2781] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:15.515158 24098 raft_consensus.cc:482] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:15.515214 24098 raft_consensus.cc:3037] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:15.516631 24098 raft_consensus.cc:504] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.517025 24098 leader_election.cc:288] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [CANDIDATE]: Term 1 election: Requested vote from peers 0a07ab2e431a4115a513dfd0afbfec14 (127.22.2.131:38959), b898fac4b2304c3c943c8d248f281913 (127.22.2.130:43787)
I20260808 04:45:15.517429 23795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af05302aeda4accb876f561fb546370" candidate_uuid: "64b801e7c43847408ce5c57d8035dc56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b898fac4b2304c3c943c8d248f281913"
I20260808 04:45:15.517417 23985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af05302aeda4accb876f561fb546370" candidate_uuid: "64b801e7c43847408ce5c57d8035dc56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a07ab2e431a4115a513dfd0afbfec14"
I20260808 04:45:15.517556 23795 raft_consensus.cc:3037] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:15.517557 23985 raft_consensus.cc:3037] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:15.518865 23985 raft_consensus.cc:2454] T 7af05302aeda4accb876f561fb546370 P 0a07ab2e431a4115a513dfd0afbfec14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b801e7c43847408ce5c57d8035dc56 in term 1.
I20260808 04:45:15.518865 23795 raft_consensus.cc:2454] T 7af05302aeda4accb876f561fb546370 P b898fac4b2304c3c943c8d248f281913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b801e7c43847408ce5c57d8035dc56 in term 1.
I20260808 04:45:15.519208 23494 leader_election.cc:302] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b801e7c43847408ce5c57d8035dc56, b898fac4b2304c3c943c8d248f281913; no voters:
I20260808 04:45:15.519379 24098 raft_consensus.cc:2781] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:15.519614 24098 raft_consensus.cc:686] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [term 1 LEADER]: Becoming Leader. State: Replica: 64b801e7c43847408ce5c57d8035dc56, State: Running, Role: LEADER
I20260808 04:45:15.519987 24098 consensus_queue.cc:237] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } }
I20260808 04:45:15.523212 23374 catalog_manager.cc:5495] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64b801e7c43847408ce5c57d8035dc56 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "64b801e7c43847408ce5c57d8035dc56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64b801e7c43847408ce5c57d8035dc56" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a07ab2e431a4115a513dfd0afbfec14" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b898fac4b2304c3c943c8d248f281913" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43787 } health_report { overall_health: UNKNOWN } } }
W20260808 04:45:15.528095 23662 tablet.cc:2369] T 7af05302aeda4accb876f561fb546370 P 64b801e7c43847408ce5c57d8035dc56: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:15.589566 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.574632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34344 (local address 127.22.2.129:46415)
0808 04:45:15.574831 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.574837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.575003 (+ 166us) server_negotiation.cc:407] Connection header received
0808 04:45:15.575179 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.575187 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.575235 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.575328 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:15.576033 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.576590 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.577448 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.577667 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.577798 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.585867 (+ 8069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:15.585883 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:15.585900 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:15.585927 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:15.587582 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.588063 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.588069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.588070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.588202 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.588500 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.588504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.588506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.588850 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:15.589094 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.589264 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.589393 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164315, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164315, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:15.618979 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.610147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34348 (local address 127.22.2.129:46415)
0808 04:45:15.610348 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.610352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.610433 (+ 81us) server_negotiation.cc:407] Connection header received
0808 04:45:15.610553 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.610555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.610600 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.610671 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:15.611454 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.611966 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.612939 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.613163 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.613240 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.615694 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:15.615697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:15.615699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:15.615721 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:15.617349 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.617766 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.617768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.617769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.617842 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.618130 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.618132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.618134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.618317 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:15.618436 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.618702 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.618798 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:15.646241 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:15.637056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.22.2.129:46415)
0808 04:45:15.637194 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:15.637200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:15.637371 (+ 171us) server_negotiation.cc:407] Connection header received
0808 04:45:15.637545 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:15.637547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:15.637589 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:15.637674 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:15.638346 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.638835 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.639673 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:15.639929 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:15.640048 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:15.642648 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:15.642651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:15.642653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:15.642706 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:15.644614 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.645077 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.645081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.645083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.645172 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:15.645523 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:15.645527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:15.645528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:15.645722 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:15.645917 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:15.646027 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 04:45:15.646120 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 04:45:15.647923 23532 tablet_service.cc:2908] Rejecting scan request for tablet 7af05302aeda4accb876f561fb546370: Uninitialized: clean time has not yet been initialized
I20260808 04:45:15.650146 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23468
I20260808 04:45:15.665195 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23671
I20260808 04:45:15.676990 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23867
I20260808 04:45:15.687381 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23335
2026-08-08T04:45:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2755 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[24121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[24121](info): set up 2 sockets
Aug 08 04:45:15 dist-test-slave-04bb krb5kdc[24121](info): commencing operation
krb5kdc: starting...
W20260808 04:45:16.983105 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:17 dist-test-slave-04bb krb5kdc[24121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:17.110834 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46041
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:46041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:17.224339 24142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:17.224644 24142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:17.224690 24142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:17.228863 24142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:17.228914 24142 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 04:45:17.228931 24142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:17.228971 24142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:17.229004 24142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:17.229035 24142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:17.234378 24142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:46041
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:17.235357 24142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:17.236474 24142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:45:17.245376 24142 server_base.cc:1034] running on GCE node
W20260808 04:45:17.245317 24155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:17.245409 24153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:17.245317 24152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:17.246127 24142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:17.247201 24142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:17.248369 24142 hybrid_clock.cc:648] HybridClock initialized: now 1786164317248351 us; error 31 us; skew 500 ppm
Aug 08 04:45:17 dist-test-slave-04bb krb5kdc[24121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:17.251173 24142 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:17.252028 24142 webserver.cc:460] Webserver started at http://127.22.2.190:45959/ using document root <none> and password file <none>
I20260808 04:45:17.252645 24142 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:17.252738 24142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:17.252997 24142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:17.255084 24142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "76944cb3298e46b4bb27c7ed4d1712f7"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "0a537b0f76ce61e3fcdf7f0ff423ad87"
server_key_iv: "29e9aa58b8f2d1f31993f38e10a5672c"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.255656 24142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "76944cb3298e46b4bb27c7ed4d1712f7"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "0a537b0f76ce61e3fcdf7f0ff423ad87"
server_key_iv: "29e9aa58b8f2d1f31993f38e10a5672c"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.259373 24142 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:45:17.261793 24164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:17.263018 24142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:17.263160 24142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "76944cb3298e46b4bb27c7ed4d1712f7"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "0a537b0f76ce61e3fcdf7f0ff423ad87"
server_key_iv: "29e9aa58b8f2d1f31993f38e10a5672c"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.263267 24142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:17.273092 24142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:17.275687 24142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:17.275888 24142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:17.283541 24142 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:46041
I20260808 04:45:17.283617 24256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:46041 every 8 connection(s)
I20260808 04:45:17.284586 24142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:17.287079 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 24142
I20260808 04:45:17.287206 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:17.287516 22538 external_mini_cluster.cc:1428] Setting key 207951255ce44bc9d6f55525de0987ad
I20260808 04:45:17.287576 24257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:17.294802 24257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7: Bootstrap starting.
Aug 08 04:45:17 dist-test-slave-04bb krb5kdc[24121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:17.297657 24257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:17.299017 24257 log.cc:824] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:17.300974 24257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7: No bootstrap required, opened a new log
I20260808 04:45:17.302363 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.288930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.22.2.190:46041)
0808 04:45:17.289380 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:17.289396 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:17.289432 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:45:17.290321 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:17.290340 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:17.290652 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:17.290928 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:17.291855 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.292912 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.294081 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.294521 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.294634 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.297622 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:17.297627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:17.297640 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:17.297667 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:17.299521 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.299982 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.299987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.299992 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.300102 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.300400 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.300403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.300405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.300877 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:17.301091 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:17.301390 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 04:45:17.301669 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":142,"threads_started":1}
I20260808 04:45:17.304409 24257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76944cb3298e46b4bb27c7ed4d1712f7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46041 } }
I20260808 04:45:17.304699 24257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:17.304770 24257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76944cb3298e46b4bb27c7ed4d1712f7, State: Initialized, Role: FOLLOWER
I20260808 04:45:17.305258 24257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76944cb3298e46b4bb27c7ed4d1712f7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46041 } }
I20260808 04:45:17.305400 24257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:17.305449 24257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:17.305521 24257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:17.306294 24257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76944cb3298e46b4bb27c7ed4d1712f7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46041 } }
I20260808 04:45:17.306602 24257 leader_election.cc:302] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76944cb3298e46b4bb27c7ed4d1712f7; no voters:
I20260808 04:45:17.306854 24257 leader_election.cc:288] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:17.307044 24264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:17.307327 24264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [term 1 LEADER]: Becoming Leader. State: Replica: 76944cb3298e46b4bb27c7ed4d1712f7, State: Running, Role: LEADER
I20260808 04:45:17.307662 24264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76944cb3298e46b4bb27c7ed4d1712f7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46041 } }
I20260808 04:45:17.308115 24257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:17.309415 24265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76944cb3298e46b4bb27c7ed4d1712f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76944cb3298e46b4bb27c7ed4d1712f7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46041 } } }
I20260808 04:45:17.309541 24265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:17.309888 24268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76944cb3298e46b4bb27c7ed4d1712f7. Latest consensus state: current_term: 1 leader_uuid: "76944cb3298e46b4bb27c7ed4d1712f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76944cb3298e46b4bb27c7ed4d1712f7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46041 } } }
I20260808 04:45:17.309995 24268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:17.309948 24282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:17.313824 24282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:17.319283 24282 catalog_manager.cc:1347] Generated new cluster ID: aafabf9262f24bfcb416c6ac61692cf3
I20260808 04:45:17.319346 24282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:17.368821 24282 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:17.369818 24282 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:17.381388 24282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7: Generated new TSK 0
I20260808 04:45:17.382130 24282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:17.423775 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46041
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:17.537237 24300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:17.537472 24300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:17.537557 24300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:17.541530 24300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:17.541581 24300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:17.541657 24300 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:17.541723 24300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:17.546787 24300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:46041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:17.547796 24300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:17.548840 24300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:17.558261 24307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:17.558303 24308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:17.558295 24311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:17.558619 24300 server_base.cc:1034] running on GCE node
I20260808 04:45:17.559021 24300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:17.559656 24300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:17.560849 24300 hybrid_clock.cc:648] HybridClock initialized: now 1786164317560827 us; error 31 us; skew 500 ppm
Aug 08 04:45:17 dist-test-slave-04bb krb5kdc[24121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:17.563548 24300 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:17.564357 24300 webserver.cc:460] Webserver started at http://127.22.2.129:37137/ using document root <none> and password file <none>
I20260808 04:45:17.564956 24300 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:17.565018 24300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:17.565191 24300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:17.567399 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "bd714916c8de43f19d54f5ab3e533e69"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "91026a81923163b7ad46018b21f38edd"
server_key_iv: "a0fe75a70a5b8bcecfc7cc733e47929c"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.567987 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "bd714916c8de43f19d54f5ab3e533e69"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "91026a81923163b7ad46018b21f38edd"
server_key_iv: "a0fe75a70a5b8bcecfc7cc733e47929c"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.572063 24300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 04:45:17.574745 24321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:17.576005 24300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 04:45:17.576128 24300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "bd714916c8de43f19d54f5ab3e533e69"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "91026a81923163b7ad46018b21f38edd"
server_key_iv: "a0fe75a70a5b8bcecfc7cc733e47929c"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.576234 24300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:17.590622 24300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:17.593606 24300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:17.593801 24300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:17.595214 24300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:17.595263 24300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:17.595330 24300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:17.595366 24300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:45:17 dist-test-slave-04bb krb5kdc[24121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:17.606706 24300 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:38811
I20260808 04:45:17.606793 24490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:38811 every 8 connection(s)
I20260808 04:45:17.607894 24300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:17.609762 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.596254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:40863 (local address 127.22.2.190:46041)
0808 04:45:17.596397 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:17.596400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:17.597454 (+ 1054us) server_negotiation.cc:407] Connection header received
0808 04:45:17.598669 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:17.598676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:17.598749 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:17.598871 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:17.600383 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.601044 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.602285 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.602525 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.602645 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.605623 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:17.605627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:17.605630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:17.605659 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:17.607386 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.608063 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.608066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.608067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.608158 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.608619 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.608623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.608625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.608953 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:17.609055 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:17.609517 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 04:45:17.609627 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:45:17.610280 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 24300
I20260808 04:45:17.610405 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:17.610599 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.596757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46041 (local address 127.22.2.129:40863)
0808 04:45:17.597309 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:17.597350 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:17.598405 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:17.599073 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:17.599082 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:17.599515 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:17.600225 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:17.600250 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.601189 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.601193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:17.602149 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:17.602165 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.602699 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.602704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:17.602873 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.603485 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:17.603510 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:17.605406 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:17.607521 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:17.607525 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:17.607536 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:17.607908 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:17.608264 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:17.608267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:17.608270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:17.608474 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:17.609063 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:17.609068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:17.609281 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 04:45:17.609432 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 04:45:17.609664 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":143,"threads_started":1}
I20260808 04:45:17.610709 22538 external_mini_cluster.cc:1428] Setting key bb2840abb81b499d876c2ba10bd9a4f7
I20260808 04:45:17.611907 24493 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46041
I20260808 04:45:17.612041 24493 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:17.612463 24493 heartbeater.cc:511] Master 127.22.2.190:46041 requested a full tablet report, sending...
I20260808 04:45:17.616266 24186 ts_manager.cc:194] Registered new tserver with Master: bd714916c8de43f19d54f5ab3e533e69 (127.22.2.129:38811)
I20260808 04:45:17.617614 24186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:40863
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:45:17.622493 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.614075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:51877 (local address 127.22.2.190:46041)
0808 04:45:17.614215 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:17.614221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:17.614562 (+ 341us) server_negotiation.cc:407] Connection header received
0808 04:45:17.614662 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:17.614667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:17.614725 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:17.614815 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:17.615313 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.616004 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.617247 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.617574 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.617686 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.619056 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:17.619060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:17.619062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:17.619088 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:17.620843 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.621293 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.621297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.621298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.621392 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.621676 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.621679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.621681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.621855 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:17.621961 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:17.622230 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 04:45:17.622350 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 04:45:17.622500 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.613964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46041 (local address 127.22.2.129:51877)
0808 04:45:17.614467 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:17.614482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:17.614575 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:17.614844 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:17.614849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:17.615071 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:17.615184 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:17.615191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.616151 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.616157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:17.617108 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:17.617123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.617714 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.617720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:17.617853 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.618587 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:17.618608 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:17.618915 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:17.620952 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:17.620957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:17.620959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:17.621158 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:17.621488 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:17.621491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:17.621493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:17.621580 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:17.621965 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:17.621969 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:17.622063 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 04:45:17.622171 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 04:45:17.622300 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:17.641674 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46041
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:17.757700 24506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:17.757970 24506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:17.758052 24506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:17.762125 24506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:17.762212 24506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:17.762291 24506 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:17.762372 24506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:17.767729 24506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:46041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:17.768878 24506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:17.769914 24506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:45:17.780010 24506 server_base.cc:1034] running on GCE node
W20260808 04:45:17.779968 24516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:17.780059 24521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:17.779948 24517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:17.780814 24506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:17.781495 24506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:17.782708 24506 hybrid_clock.cc:648] HybridClock initialized: now 1786164317782671 us; error 57 us; skew 500 ppm
Aug 08 04:45:17 dist-test-slave-04bb krb5kdc[24121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:17.785842 24506 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:17.786820 24506 webserver.cc:460] Webserver started at http://127.22.2.130:37469/ using document root <none> and password file <none>
I20260808 04:45:17.787428 24506 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:17.787549 24506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:17.787809 24506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:17.790040 24506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "f5de29c3a1a94f6c8f0924b081072aaa"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "96c801fbbaf582d07e84f0e54c546d81"
server_key_iv: "c5fd89bacb659e5d57381f0eb3447da0"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.790660 24506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "f5de29c3a1a94f6c8f0924b081072aaa"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "96c801fbbaf582d07e84f0e54c546d81"
server_key_iv: "c5fd89bacb659e5d57381f0eb3447da0"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.794724 24506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:45:17.797606 24533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:17.798870 24506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:17.799014 24506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "f5de29c3a1a94f6c8f0924b081072aaa"
format_stamp: "Formatted at 2026-08-08 04:45:17 on dist-test-slave-04bb"
server_key: "96c801fbbaf582d07e84f0e54c546d81"
server_key_iv: "c5fd89bacb659e5d57381f0eb3447da0"
server_key_version: "encryptionkey@0"
I20260808 04:45:17.799161 24506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:17.811067 24506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:17.814639 24506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:17.814844 24506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:17.816372 24506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:17.816447 24506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:17.816519 24506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:17.816586 24506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 08 04:45:17 dist-test-slave-04bb krb5kdc[24121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:17.828369 24506 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41925
I20260808 04:45:17.828404 24702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41925 every 8 connection(s)
I20260808 04:45:17.829372 24506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:17.831975 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.817072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:43287 (local address 127.22.2.190:46041)
0808 04:45:17.817243 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:17.817250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:17.818721 (+ 1471us) server_negotiation.cc:407] Connection header received
0808 04:45:17.820283 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:17.820287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:17.820366 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:17.820471 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:17.822365 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.822964 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.824932 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.825140 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.825231 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.828040 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:17.828044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:17.828047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:17.828076 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:17.829671 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.830434 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.830437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.830438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.830521 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.830876 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.830880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.830882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.831074 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:17.831188 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:17.831671 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 04:45:17.831793 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:45:17.832782 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.818023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46041 (local address 127.22.2.130:43287)
0808 04:45:17.818525 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:17.818595 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:17.819948 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:17.820760 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:17.820774 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:17.821357 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:17.822168 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:17.822193 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.823100 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.823116 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:17.824804 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:17.824816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.825337 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.825344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:17.825570 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.826370 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:17.826394 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:17.827808 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:17.829789 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:17.829795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:17.829809 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:17.830162 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:17.830631 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:17.830634 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:17.830636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:17.830777 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:17.831186 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:17.831193 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:17.831436 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 04:45:17.831623 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 04:45:17.831929 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":110,"threads_started":1}
I20260808 04:45:17.834084 24703 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46041
I20260808 04:45:17.834244 24703 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:17.834738 24703 heartbeater.cc:511] Master 127.22.2.190:46041 requested a full tablet report, sending...
I20260808 04:45:17.835837 24188 ts_manager.cc:194] Registered new tserver with Master: f5de29c3a1a94f6c8f0924b081072aaa (127.22.2.130:41925)
I20260808 04:45:17.836660 24188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:43287
I20260808 04:45:17.838197 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 24506
I20260808 04:45:17.838303 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:17.838564 22538 external_mini_cluster.cc:1428] Setting key bce22bd190dfa8fa54aedacf667e47ab
I20260808 04:45:17.843850 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.836326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46041 (local address 127.22.2.130:58207)
0808 04:45:17.836649 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:17.836663 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:17.836776 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:17.837068 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:17.837071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:17.837274 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:17.837384 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:17.837390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.838404 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.838407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:17.839179 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:17.839188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.839639 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.839643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:17.839753 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.840221 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:17.840232 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:17.840585 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:17.842249 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:17.842256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:17.842260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:17.842533 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:17.842821 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:17.842826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:17.842830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:17.842935 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:17.843374 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:17.843379 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:17.843489 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 04:45:17.843603 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 04:45:17.843703 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":114,"threads_started":1}
I20260808 04:45:17.843855 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:17.836420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:58207 (local address 127.22.2.190:46041)
0808 04:45:17.836560 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:17.836565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:17.836769 (+ 204us) server_negotiation.cc:407] Connection header received
0808 04:45:17.836872 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:17.836875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:17.836952 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:17.837033 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:17.837502 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.838245 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.839296 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:17.839521 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:17.839636 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:17.840732 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:17.840736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:17.840739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:17.840762 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:17.842081 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.842638 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.842641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.842642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.842719 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:17.843034 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:17.843036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:17.843037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:17.843271 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:17.843351 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:17.843662 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 04:45:17.843751 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:17.868358 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46041
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:17.983444 24714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:17.983693 24714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:17.983736 24714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:17.987836 24714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:17.987893 24714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:17.987931 24714 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:17.987998 24714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:17.993283 24714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:46041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:17.994304 24714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:17.995321 24714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:18.005055 24726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:18.005062 24723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:18.005486 24724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:18.005820 24714 server_base.cc:1034] running on GCE node
I20260808 04:45:18.006266 24714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:18.006922 24714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:18.008112 24714 hybrid_clock.cc:648] HybridClock initialized: now 1786164318008097 us; error 29 us; skew 500 ppm
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[24121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164318, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:18.011138 24714 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:18.012092 24714 webserver.cc:460] Webserver started at http://127.22.2.131:34325/ using document root <none> and password file <none>
I20260808 04:45:18.012773 24714 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:18.012876 24714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:18.013134 24714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:18.015317 24714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "619fa13ef56742af9bb40e2b01b08a5f"
format_stamp: "Formatted at 2026-08-08 04:45:18 on dist-test-slave-04bb"
server_key: "d5bc921fd9928f60f9ed8e057d56c702"
server_key_iv: "2736d142d70d0b4fea1319ee3e12727b"
server_key_version: "encryptionkey@0"
I20260808 04:45:18.015905 24714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "619fa13ef56742af9bb40e2b01b08a5f"
format_stamp: "Formatted at 2026-08-08 04:45:18 on dist-test-slave-04bb"
server_key: "d5bc921fd9928f60f9ed8e057d56c702"
server_key_iv: "2736d142d70d0b4fea1319ee3e12727b"
server_key_version: "encryptionkey@0"
I20260808 04:45:18.019789 24714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 04:45:18.022778 24733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:18.024140 24714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:18.024294 24714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "619fa13ef56742af9bb40e2b01b08a5f"
format_stamp: "Formatted at 2026-08-08 04:45:18 on dist-test-slave-04bb"
server_key: "d5bc921fd9928f60f9ed8e057d56c702"
server_key_iv: "2736d142d70d0b4fea1319ee3e12727b"
server_key_version: "encryptionkey@0"
I20260808 04:45:18.024436 24714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:18.037647 24714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:18.041076 24714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:18.041283 24714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:18.042791 24714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:18.042873 24714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:18.042941 24714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:18.042985 24714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[24121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164318, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:18.054270 24714 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:46729
I20260808 04:45:18.054580 24912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:46729 every 8 connection(s)
I20260808 04:45:18.055464 24714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:18.057945 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.043631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:45999 (local address 127.22.2.190:46041)
0808 04:45:18.043763 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.043767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.044935 (+ 1168us) server_negotiation.cc:407] Connection header received
0808 04:45:18.046034 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.046038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.046134 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.046186 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:18.047738 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.048335 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.050283 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.050687 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.050824 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.053778 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:18.053781 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:18.053783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:18.053851 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:18.055702 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.056347 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.056350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.056352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.056442 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.056821 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.056824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.056826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.057019 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:18.057135 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.057643 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.057770 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 04:45:18.058665 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.044168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46041 (local address 127.22.2.131:45999)
0808 04:45:18.044781 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.044825 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.045763 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.046515 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.046521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.046957 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:18.047567 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.047604 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.048484 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.048489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.050080 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.050095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.051017 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.051020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.051170 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.051763 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:18.051779 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:18.053584 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:18.055823 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:18.055828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:18.055840 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:18.056242 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:18.056560 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:18.056563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:18.056565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:18.056708 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:18.057138 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:18.057147 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:18.057384 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.057568 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.057878 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":151,"threads_started":1}
I20260808 04:45:18.060029 24913 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46041
I20260808 04:45:18.060194 24913 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:18.060883 24913 heartbeater.cc:511] Master 127.22.2.190:46041 requested a full tablet report, sending...
I20260808 04:45:18.061983 24186 ts_manager.cc:194] Registered new tserver with Master: 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729)
I20260808 04:45:18.062744 24186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:45999
I20260808 04:45:18.065120 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 24714
I20260808 04:45:18.065267 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:18.065562 22538 external_mini_cluster.cc:1428] Setting key ff96b835f3b8a54ad3c7a42f577ced28
I20260808 04:45:18.068344 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:45:18.070611 24920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.062382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46041 (local address 127.22.2.131:35939)
0808 04:45:18.062753 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.062766 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.062900 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.063252 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.063254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.063473 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:18.063587 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.063594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.064385 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.064389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.065095 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.065105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.065709 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.065712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.065781 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.066361 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:18.066379 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:18.066831 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:18.069139 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:18.069141 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:18.069143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:18.069335 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:18.069666 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:18.069668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:18.069669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:18.069743 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:18.070217 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:18.070219 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:18.070293 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.070377 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.070481 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":113,"threads_started":1}
I20260808 04:45:18.070924 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.062469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35939 (local address 127.22.2.190:46041)
0808 04:45:18.062612 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.062616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.062850 (+ 234us) server_negotiation.cc:407] Connection header received
0808 04:45:18.063009 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.063012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.063067 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.063212 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:18.063729 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.064241 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.065241 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.065552 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.065672 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.066990 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:18.066993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:18.066995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:18.067045 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:18.069026 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.069442 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.069448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.069453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.069555 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.069839 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.069846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.069851 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.070115 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:18.070232 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.070474 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.070653 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[24121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164318, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[24121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164318, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:18.099310 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.088869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.22.2.190:46041)
0808 04:45:18.089007 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.089011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.089024 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:45:18.089069 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.089072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.089112 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.089225 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:18.089951 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.090523 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.091337 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.091557 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.091696 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.095368 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:18.095372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:18.095375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:18.095409 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:18.097162 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.097693 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.097697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.097699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.097826 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.098206 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.098210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.098212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.098470 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:18.098637 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.099013 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.099136 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:45:18.102998 24186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:18.106281 24186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:18.121706 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.115508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:38811 (local address 127.0.0.1:41450)
0808 04:45:18.116266 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.116281 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.116375 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.117333 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.117336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.117355 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:18.117474 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.117480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.118833 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.118840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.120098 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.120109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.121111 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.121114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.121245 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.121290 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.121422 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.121545 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":199,"thread_start_us":98,"threads_started":1}
I20260808 04:45:18.121706 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.116475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.22.2.129:38811)
0808 04:45:18.116865 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.116873 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.116898 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:45:18.116971 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.116976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.117161 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.117355 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:18.117599 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.118676 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.120256 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.120975 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.121112 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.121264 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.121455 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.121548 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":128,"threads_started":1}
I20260808 04:45:18.122587 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.116230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.22.2.130:41925)
0808 04:45:18.116672 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.116683 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.117033 (+ 350us) server_negotiation.cc:407] Connection header received
0808 04:45:18.117261 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.117269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.117452 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.117580 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:18.117984 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.118716 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.120505 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.121370 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.121444 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.121593 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.122326 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.122449 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":100,"threads_started":1}
I20260808 04:45:18.122617 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.115301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:46729 (local address 127.0.0.1:43796)
0808 04:45:18.116053 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.116117 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.116261 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.117100 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.117106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.117130 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:18.117292 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.117302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.119062 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.119065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.121131 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.121143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.122202 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.122205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.122321 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.122345 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.122420 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.122486 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":74,"thread_start_us":80,"threads_started":1}
I20260808 04:45:18.122686 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.115465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.22.2.131:46729)
0808 04:45:18.115892 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.115905 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.116414 (+ 509us) server_negotiation.cc:407] Connection header received
0808 04:45:18.116501 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.116509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.116768 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.116931 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:18.117423 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.118676 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.121279 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.122072 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.122187 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.122377 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.122455 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.122536 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":141,"threads_started":1}
I20260808 04:45:18.123294 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.116040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41925 (local address 127.0.0.1:55614)
0808 04:45:18.116934 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.116947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.117104 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.117786 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.117788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.117810 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:18.117872 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.117875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.118865 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.118867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.120369 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.120382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.121693 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.121696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.122130 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.122168 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.123061 (+ 893us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.123160 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":170,"thread_start_us":95,"threads_started":1}
I20260808 04:45:18.125169 24397 tablet_service.cc:1467] Processing CreateTablet for tablet 113174dfcb2c44608338b397427565c5 (DEFAULT_TABLE table=test-table [id=9d640d70782d410b88e8f82410c35fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:18.126376 24397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 113174dfcb2c44608338b397427565c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.126012 24612 tablet_service.cc:1467] Processing CreateTablet for tablet 113174dfcb2c44608338b397427565c5 (DEFAULT_TABLE table=test-table [id=9d640d70782d410b88e8f82410c35fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:18.127013 24612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 113174dfcb2c44608338b397427565c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.135090 24941 tablet_bootstrap.cc:492] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Bootstrap starting.
I20260808 04:45:18.134768 24819 tablet_service.cc:1467] Processing CreateTablet for tablet 113174dfcb2c44608338b397427565c5 (DEFAULT_TABLE table=test-table [id=9d640d70782d410b88e8f82410c35fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:18.135910 24819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 113174dfcb2c44608338b397427565c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.137277 24941 tablet_bootstrap.cc:654] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.138106 24941 log.cc:824] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:18.138167 24942 tablet_bootstrap.cc:492] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: Bootstrap starting.
I20260808 04:45:18.140692 24941 tablet_bootstrap.cc:492] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: No bootstrap required, opened a new log
I20260808 04:45:18.140839 24942 tablet_bootstrap.cc:654] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.140923 24941 ts_tablet_manager.cc:1397] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:45:18.141703 24942 log.cc:824] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:18.143180 24946 tablet_bootstrap.cc:492] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Bootstrap starting.
I20260808 04:45:18.143702 24942 tablet_bootstrap.cc:492] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: No bootstrap required, opened a new log
I20260808 04:45:18.143901 24942 ts_tablet_manager.cc:1397] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:45:18.145079 24941 raft_consensus.cc:348] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.145355 24941 raft_consensus.cc:374] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.145378 24946 tablet_bootstrap.cc:654] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.145414 24941 raft_consensus.cc:729] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5de29c3a1a94f6c8f0924b081072aaa, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.145871 24941 consensus_queue.cc:260] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.146121 24946 log.cc:824] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:18.146624 24703 heartbeater.cc:503] Master 127.22.2.190:46041 was elected leader, sending a full tablet report...
I20260808 04:45:18.146888 24941 ts_tablet_manager.cc:1428] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:45:18.148169 24942 raft_consensus.cc:348] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.148442 24942 raft_consensus.cc:374] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.148504 24942 raft_consensus.cc:729] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd714916c8de43f19d54f5ab3e533e69, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.148792 24946 tablet_bootstrap.cc:492] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: No bootstrap required, opened a new log
I20260808 04:45:18.148998 24946 ts_tablet_manager.cc:1397] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:45:18.148962 24942 consensus_queue.cc:260] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.149757 24493 heartbeater.cc:503] Master 127.22.2.190:46041 was elected leader, sending a full tablet report...
I20260808 04:45:18.149906 24942 ts_tablet_manager.cc:1428] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 04:45:18.152453 24946 raft_consensus.cc:348] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.152711 24946 raft_consensus.cc:374] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.152778 24946 raft_consensus.cc:729] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619fa13ef56742af9bb40e2b01b08a5f, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.153242 24946 consensus_queue.cc:260] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.153887 24913 heartbeater.cc:503] Master 127.22.2.190:46041 was elected leader, sending a full tablet report...
I20260808 04:45:18.154142 24946 ts_tablet_manager.cc:1428] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 04:45:18.279013 24953 raft_consensus.cc:482] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:18.279349 24953 raft_consensus.cc:504] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.280659 24953 leader_election.cc:288] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729), f5de29c3a1a94f6c8f0924b081072aaa (127.22.2.130:41925)
I20260808 04:45:18.285069 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.280897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41925 (local address 127.22.2.129:56667)
0808 04:45:18.281431 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.281443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.281581 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.281858 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.281861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.281878 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:18.282027 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.282033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.282758 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.282764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.283727 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.283738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.284638 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.284642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.284750 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.284777 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.284851 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.284912 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":81,"thread_start_us":155,"threads_started":1}
I20260808 04:45:18.285111 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.281028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56667 (local address 127.22.2.130:41925)
0808 04:45:18.281223 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.281229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.281531 (+ 302us) server_negotiation.cc:407] Connection header received
0808 04:45:18.281701 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.281705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.281759 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.281847 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:18.282149 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.282644 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.283859 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.284473 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.284628 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.284706 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.284883 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.284959 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260808 04:45:18.285337 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.281041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:43547 (local address 127.22.2.131:46729)
0808 04:45:18.281199 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.281204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.281459 (+ 255us) server_negotiation.cc:407] Connection header received
0808 04:45:18.281630 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.281634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.281686 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.281772 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:18.282090 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.282602 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.283981 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.284616 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.284749 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.284785 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.285114 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.285204 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:45:18.285722 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.280897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:46729 (local address 127.22.2.129:43547)
0808 04:45:18.281357 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.281369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.281505 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.281786 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.281788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.281804 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:18.281972 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.281978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.282718 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.282723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.283797 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.283809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.284867 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.284870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.284967 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.284990 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.285484 (+ 494us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.285556 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":51,"thread_start_us":149,"threads_started":1}
I20260808 04:45:18.286326 24634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "113174dfcb2c44608338b397427565c5" candidate_uuid: "bd714916c8de43f19d54f5ab3e533e69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" is_pre_election: true
I20260808 04:45:18.286661 24634 raft_consensus.cc:2454] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd714916c8de43f19d54f5ab3e533e69 in term 0.
I20260808 04:45:18.286648 24850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "113174dfcb2c44608338b397427565c5" candidate_uuid: "bd714916c8de43f19d54f5ab3e533e69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619fa13ef56742af9bb40e2b01b08a5f" is_pre_election: true
I20260808 04:45:18.286916 24850 raft_consensus.cc:2454] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd714916c8de43f19d54f5ab3e533e69 in term 0.
I20260808 04:45:18.287189 24327 leader_election.cc:302] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd714916c8de43f19d54f5ab3e533e69, f5de29c3a1a94f6c8f0924b081072aaa; no voters:
I20260808 04:45:18.287487 24953 raft_consensus.cc:2781] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:18.287591 24953 raft_consensus.cc:482] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:18.287638 24953 raft_consensus.cc:3037] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.288991 24953 raft_consensus.cc:504] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.289376 24953 leader_election.cc:288] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [CANDIDATE]: Term 1 election: Requested vote from peers 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729), f5de29c3a1a94f6c8f0924b081072aaa (127.22.2.130:41925)
I20260808 04:45:18.289714 24850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "113174dfcb2c44608338b397427565c5" candidate_uuid: "bd714916c8de43f19d54f5ab3e533e69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619fa13ef56742af9bb40e2b01b08a5f"
I20260808 04:45:18.289739 24634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "113174dfcb2c44608338b397427565c5" candidate_uuid: "bd714916c8de43f19d54f5ab3e533e69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5de29c3a1a94f6c8f0924b081072aaa"
I20260808 04:45:18.289845 24634 raft_consensus.cc:3037] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.289845 24850 raft_consensus.cc:3037] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.291565 24850 raft_consensus.cc:2454] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd714916c8de43f19d54f5ab3e533e69 in term 1.
I20260808 04:45:18.291565 24634 raft_consensus.cc:2454] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd714916c8de43f19d54f5ab3e533e69 in term 1.
I20260808 04:45:18.291875 24327 leader_election.cc:302] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd714916c8de43f19d54f5ab3e533e69, f5de29c3a1a94f6c8f0924b081072aaa; no voters:
I20260808 04:45:18.292091 24953 raft_consensus.cc:2781] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:18.292330 24953 raft_consensus.cc:686] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 1 LEADER]: Becoming Leader. State: Replica: bd714916c8de43f19d54f5ab3e533e69, State: Running, Role: LEADER
I20260808 04:45:18.292729 24953 consensus_queue.cc:237] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } }
I20260808 04:45:18.295939 24188 catalog_manager.cc:5495] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd714916c8de43f19d54f5ab3e533e69 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "bd714916c8de43f19d54f5ab3e533e69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } health_report { overall_health: UNKNOWN } } }
W20260808 04:45:18.306519 24915 tablet.cc:2369] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:18.324633 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.320691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.22.2.129:38811)
0808 04:45:18.320844 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.320851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.320872 (+ 21us) server_negotiation.cc:407] Connection header received
0808 04:45:18.321139 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.321143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.321204 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.321322 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:18.321582 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.322093 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.323004 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.323230 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.323353 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.323696 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:18.324100 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:18.324244 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.324385 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.324442 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 04:45:18.330657 24704 tablet.cc:2369] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:18.330927 24634 raft_consensus.cc:1261] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Refusing update from remote peer bd714916c8de43f19d54f5ab3e533e69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:18.331060 24850 raft_consensus.cc:1261] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Refusing update from remote peer bd714916c8de43f19d54f5ab3e533e69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:18.331701 24953 consensus_queue.cc:1035] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:18.331952 24959 consensus_queue.cc:1035] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:18.339672 24965 mvcc.cc:205] Tried to move back new op lower bound from 7316129047876120576 to 7316129047730044928. Current Snapshot: MvccSnapshot[applied={T|T < 7316129047876120576}]
I20260808 04:45:18.340654 24967 mvcc.cc:205] Tried to move back new op lower bound from 7316129047876120576 to 7316129047730044928. Current Snapshot: MvccSnapshot[applied={T|T < 7316129047876120576}]
I20260808 04:45:18.341249 24966 mvcc.cc:205] Tried to move back new op lower bound from 7316129047876120576 to 7316129047730044928. Current Snapshot: MvccSnapshot[applied={T|T < 7316129047876120576}]
I20260808 04:45:18.353981 24186 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:18.354167 24186 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:18.356982 24186 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7: Sending DeleteTablet for 3 replicas of tablet 113174dfcb2c44608338b397427565c5
I20260808 04:45:18.357962 24612 tablet_service.cc:1514] Processing DeleteTablet for tablet 113174dfcb2c44608338b397427565c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:55614
I20260808 04:45:18.357990 24819 tablet_service.cc:1514] Processing DeleteTablet for tablet 113174dfcb2c44608338b397427565c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:43796
I20260808 04:45:18.358187 24397 tablet_service.cc:1514] Processing DeleteTablet for tablet 113174dfcb2c44608338b397427565c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:41450
I20260808 04:45:18.358492 24983 tablet_replica.cc:330] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: stopping tablet replica
I20260808 04:45:18.358642 24985 tablet_replica.cc:330] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: stopping tablet replica
I20260808 04:45:18.358806 24983 raft_consensus.cc:2229] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:18.358938 24985 raft_consensus.cc:2229] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:18.359126 24983 raft_consensus.cc:2258] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:18.359593 24186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 04:45:18.359674 24981 tablet_replica.cc:330] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: stopping tablet replica
I20260808 04:45:18.359833 24985 raft_consensus.cc:2258] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 04:45:18.360011 24186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:18.360371 24981 raft_consensus.cc:2229] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:18.360749 24981 raft_consensus.cc:2258] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:18.361454 24985 ts_tablet_manager.cc:1905] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:18.362149 24981 ts_tablet_manager.cc:1905] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:18.364562 24985 ts_tablet_manager.cc:1918] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:18.364671 24985 log.cc:1196] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/113174dfcb2c44608338b397427565c5
I20260808 04:45:18.364831 24981 ts_tablet_manager.cc:1918] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:18.364940 24981 log.cc:1196] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/113174dfcb2c44608338b397427565c5
I20260808 04:45:18.365011 24985 ts_tablet_manager.cc:1939] T 113174dfcb2c44608338b397427565c5 P bd714916c8de43f19d54f5ab3e533e69: Deleting consensus metadata
I20260808 04:45:18.365255 24981 ts_tablet_manager.cc:1939] T 113174dfcb2c44608338b397427565c5 P f5de29c3a1a94f6c8f0924b081072aaa: Deleting consensus metadata
I20260808 04:45:18.366573 24168 catalog_manager.cc:4841] TS f5de29c3a1a94f6c8f0924b081072aaa (127.22.2.130:41925): tablet 113174dfcb2c44608338b397427565c5 (table test-table [id=9d640d70782d410b88e8f82410c35fce]) successfully deleted
I20260808 04:45:18.366866 24168 catalog_manager.cc:4841] TS bd714916c8de43f19d54f5ab3e533e69 (127.22.2.129:38811): tablet 113174dfcb2c44608338b397427565c5 (table test-table [id=9d640d70782d410b88e8f82410c35fce]) successfully deleted
I20260808 04:45:18.366895 24612 tablet_service.cc:1467] Processing CreateTablet for tablet 5f14fdbb886f4a9ba96794dc0295ebdd (DEFAULT_TABLE table=test-table [id=93c6480b8c044884bfc93db16c72fe21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:18.367133 24612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f14fdbb886f4a9ba96794dc0295ebdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.367341 24397 tablet_service.cc:1467] Processing CreateTablet for tablet 5f14fdbb886f4a9ba96794dc0295ebdd (DEFAULT_TABLE table=test-table [id=93c6480b8c044884bfc93db16c72fe21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:18.367453 24819 tablet_service.cc:1467] Processing CreateTablet for tablet 5f14fdbb886f4a9ba96794dc0295ebdd (DEFAULT_TABLE table=test-table [id=93c6480b8c044884bfc93db16c72fe21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:18.367575 24397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f14fdbb886f4a9ba96794dc0295ebdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.367707 24819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f14fdbb886f4a9ba96794dc0295ebdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.369342 24941 tablet_bootstrap.cc:492] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: Bootstrap starting.
I20260808 04:45:18.369856 24942 tablet_bootstrap.cc:492] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: Bootstrap starting.
I20260808 04:45:18.370298 24946 tablet_bootstrap.cc:492] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: Bootstrap starting.
I20260808 04:45:18.370335 24941 tablet_bootstrap.cc:654] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.370901 24942 tablet_bootstrap.cc:654] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.371241 24946 tablet_bootstrap.cc:654] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.371527 24983 ts_tablet_manager.cc:1905] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:18.372409 24946 tablet_bootstrap.cc:492] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: No bootstrap required, opened a new log
I20260808 04:45:18.372489 24946 ts_tablet_manager.cc:1397] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:18.372949 24946 raft_consensus.cc:348] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.373011 24941 tablet_bootstrap.cc:492] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: No bootstrap required, opened a new log
I20260808 04:45:18.373054 24946 raft_consensus.cc:374] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.373085 24946 raft_consensus.cc:729] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619fa13ef56742af9bb40e2b01b08a5f, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.373087 24941 ts_tablet_manager.cc:1397] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 04:45:18.373214 24946 consensus_queue.cc:260] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.373485 24946 ts_tablet_manager.cc:1428] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:45:18.373544 24941 raft_consensus.cc:348] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.373667 24941 raft_consensus.cc:374] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.373700 24941 raft_consensus.cc:729] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5de29c3a1a94f6c8f0924b081072aaa, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.373821 24941 consensus_queue.cc:260] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.374099 24941 ts_tablet_manager.cc:1428] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:18.375979 24983 ts_tablet_manager.cc:1918] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:18.376111 24983 log.cc:1196] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/113174dfcb2c44608338b397427565c5
I20260808 04:45:18.376463 24983 ts_tablet_manager.cc:1939] T 113174dfcb2c44608338b397427565c5 P 619fa13ef56742af9bb40e2b01b08a5f: Deleting consensus metadata
I20260808 04:45:18.377624 24167 catalog_manager.cc:4841] TS 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729): tablet 113174dfcb2c44608338b397427565c5 (table test-table [id=9d640d70782d410b88e8f82410c35fce]) successfully deleted
I20260808 04:45:18.378147 24942 tablet_bootstrap.cc:492] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: No bootstrap required, opened a new log
I20260808 04:45:18.378221 24942 ts_tablet_manager.cc:1397] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 04:45:18.378580 24942 raft_consensus.cc:348] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.378679 24942 raft_consensus.cc:374] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.378700 24942 raft_consensus.cc:729] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd714916c8de43f19d54f5ab3e533e69, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.378834 24942 consensus_queue.cc:260] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.379101 24942 ts_tablet_manager.cc:1428] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:45:18.449792 24951 raft_consensus.cc:482] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:18.449967 24951 raft_consensus.cc:504] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.451220 24951 leader_election.cc:288] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd714916c8de43f19d54f5ab3e533e69 (127.22.2.129:38811), 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729)
I20260808 04:45:18.455950 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.451578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:47207 (local address 127.22.2.129:38811)
0808 04:45:18.451704 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.451709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.451986 (+ 277us) server_negotiation.cc:407] Connection header received
0808 04:45:18.452213 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.452220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.452282 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.452368 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:18.452736 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.453500 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.454918 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.455312 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.455448 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.455509 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.455736 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.455825 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 04:45:18.456037 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.451464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:46729 (local address 127.22.2.130:37337)
0808 04:45:18.451990 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.452006 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.452151 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.452494 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.452500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.452514 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:18.452651 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.452660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.453423 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.453428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.454700 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.454711 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.455606 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.455610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.455726 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.455747 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.455816 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.455881 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":257,"thread_start_us":65,"threads_started":1}
I20260808 04:45:18.456069 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.451644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:37337 (local address 127.22.2.131:46729)
0808 04:45:18.451775 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.451780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.452127 (+ 347us) server_negotiation.cc:407] Connection header received
0808 04:45:18.452298 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.452302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.452359 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.452437 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:18.452776 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.453270 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.454846 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.455502 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.455633 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.455686 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.455848 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.455940 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 04:45:18.456326 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.451464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:38811 (local address 127.22.2.130:47207)
0808 04:45:18.451862 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.451879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.452025 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.452382 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.452385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.452401 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:18.452581 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.452589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.453621 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.453625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.454748 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.454759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.455450 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.455454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.455587 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.455611 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.456091 (+ 480us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.456187 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260808 04:45:18.456844 24850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f14fdbb886f4a9ba96794dc0295ebdd" candidate_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619fa13ef56742af9bb40e2b01b08a5f" is_pre_election: true
I20260808 04:45:18.456898 24428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f14fdbb886f4a9ba96794dc0295ebdd" candidate_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd714916c8de43f19d54f5ab3e533e69" is_pre_election: true
I20260808 04:45:18.457011 24850 raft_consensus.cc:2454] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5de29c3a1a94f6c8f0924b081072aaa in term 0.
I20260808 04:45:18.457100 24428 raft_consensus.cc:2454] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5de29c3a1a94f6c8f0924b081072aaa in term 0.
I20260808 04:45:18.457496 24535 leader_election.cc:302] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 619fa13ef56742af9bb40e2b01b08a5f, f5de29c3a1a94f6c8f0924b081072aaa; no voters:
I20260808 04:45:18.457785 24951 raft_consensus.cc:2781] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:18.457859 24951 raft_consensus.cc:482] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:18.457908 24951 raft_consensus.cc:3037] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.458912 24951 raft_consensus.cc:504] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.459295 24951 leader_election.cc:288] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [CANDIDATE]: Term 1 election: Requested vote from peers bd714916c8de43f19d54f5ab3e533e69 (127.22.2.129:38811), 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729)
I20260808 04:45:18.459618 24850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f14fdbb886f4a9ba96794dc0295ebdd" candidate_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619fa13ef56742af9bb40e2b01b08a5f"
I20260808 04:45:18.459754 24850 raft_consensus.cc:3037] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.460948 24850 raft_consensus.cc:2454] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5de29c3a1a94f6c8f0924b081072aaa in term 1.
I20260808 04:45:18.461249 24428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f14fdbb886f4a9ba96794dc0295ebdd" candidate_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd714916c8de43f19d54f5ab3e533e69"
I20260808 04:45:18.461297 24535 leader_election.cc:302] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 619fa13ef56742af9bb40e2b01b08a5f, f5de29c3a1a94f6c8f0924b081072aaa; no voters:
I20260808 04:45:18.461371 24428 raft_consensus.cc:3037] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.461458 24951 raft_consensus.cc:2781] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:18.461681 24951 raft_consensus.cc:686] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 1 LEADER]: Becoming Leader. State: Replica: f5de29c3a1a94f6c8f0924b081072aaa, State: Running, Role: LEADER
I20260808 04:45:18.461932 24951 consensus_queue.cc:237] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.462565 24428 raft_consensus.cc:2454] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5de29c3a1a94f6c8f0924b081072aaa in term 1.
I20260808 04:45:18.464501 24186 catalog_manager.cc:5495] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa reported cstate change: term changed from 0 to 1, leader changed from <none> to f5de29c3a1a94f6c8f0924b081072aaa (127.22.2.130). New cstate: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } health_report { overall_health: UNKNOWN } } }
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[24121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:18.499842 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.490450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.22.2.190:46041)
0808 04:45:18.490617 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.490622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.490768 (+ 146us) server_negotiation.cc:407] Connection header received
0808 04:45:18.490868 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.490872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.490926 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.491034 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:18.491784 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.492633 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.493500 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.493841 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.493962 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.496496 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:18.496499 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:18.496501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:18.496578 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:18.498006 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.498566 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.498568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.498570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.498642 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.498920 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.498923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.498924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.499084 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:18.499159 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.499482 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.499693 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":64}
I20260808 04:45:18.499863 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.490325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46041 (local address 127.0.0.1:32794)
0808 04:45:18.490651 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.490668 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.490788 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.491093 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.491096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.491533 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:18.491638 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.491644 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.492764 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.492767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.493365 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.493373 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.493959 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.493962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.494077 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.494760 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:18.494772 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:18.496351 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:18.498150 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:18.498160 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:18.498167 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:18.498466 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:18.498734 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:18.498737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:18.498739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:18.498834 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:18.499184 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:18.499190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:18.499312 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.499426 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.499622 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":34,"thread_start_us":139,"threads_started":1}
I20260808 04:45:18.504381 24186 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:32794:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 04:45:18.505259 24186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:18.510735 24397 tablet_service.cc:1467] Processing CreateTablet for tablet d82ee1212fca49d0844bdc6e54169c55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75c58d8df34343459de0aa230876b6ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:18.510675 24612 tablet_service.cc:1467] Processing CreateTablet for tablet d82ee1212fca49d0844bdc6e54169c55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75c58d8df34343459de0aa230876b6ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:18.511145 24397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82ee1212fca49d0844bdc6e54169c55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.511150 24612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82ee1212fca49d0844bdc6e54169c55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.514055 24942 tablet_bootstrap.cc:492] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: Bootstrap starting.
I20260808 04:45:18.514523 24941 tablet_bootstrap.cc:492] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: Bootstrap starting.
I20260808 04:45:18.514508 24819 tablet_service.cc:1467] Processing CreateTablet for tablet d82ee1212fca49d0844bdc6e54169c55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75c58d8df34343459de0aa230876b6ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:18.514891 24819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82ee1212fca49d0844bdc6e54169c55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:18.514945 24942 tablet_bootstrap.cc:654] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.515427 24941 tablet_bootstrap.cc:654] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.516917 24941 tablet_bootstrap.cc:492] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: No bootstrap required, opened a new log
I20260808 04:45:18.517006 24941 ts_tablet_manager.cc:1397] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 04:45:18.517455 24941 raft_consensus.cc:348] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.517576 24941 raft_consensus.cc:374] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.517616 24941 raft_consensus.cc:729] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5de29c3a1a94f6c8f0924b081072aaa, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.517740 24941 consensus_queue.cc:260] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.518002 24941 ts_tablet_manager.cc:1428] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:45:18.518350 24951 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.518491 24951 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:18.518580 24946 tablet_bootstrap.cc:492] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: Bootstrap starting.
I20260808 04:45:18.519253 24942 tablet_bootstrap.cc:492] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: No bootstrap required, opened a new log
I20260808 04:45:18.519369 24942 ts_tablet_manager.cc:1397] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 04:45:18.519531 24946 tablet_bootstrap.cc:654] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:18.519785 24942 raft_consensus.cc:348] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.519899 24942 raft_consensus.cc:374] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.519939 24942 raft_consensus.cc:729] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd714916c8de43f19d54f5ab3e533e69, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.520079 24942 consensus_queue.cc:260] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.520383 24942 ts_tablet_manager.cc:1428] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:18.520608 24953 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.520720 24953 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:18.520907 24946 tablet_bootstrap.cc:492] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: No bootstrap required, opened a new log
I20260808 04:45:18.520978 24946 ts_tablet_manager.cc:1397] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:18.521440 24946 raft_consensus.cc:348] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.521557 24946 raft_consensus.cc:374] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:18.521591 24946 raft_consensus.cc:729] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619fa13ef56742af9bb40e2b01b08a5f, State: Initialized, Role: FOLLOWER
I20260808 04:45:18.521737 24946 consensus_queue.cc:260] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.522006 24946 ts_tablet_manager.cc:1428] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:45:18.522018 24955 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.522210 24955 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:18.523371 24951 raft_consensus.cc:482] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:18.523463 24951 raft_consensus.cc:504] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.523840 24951 leader_election.cc:288] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd714916c8de43f19d54f5ab3e533e69 (127.22.2.129:38811), 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729)
I20260808 04:45:18.524132 24428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82ee1212fca49d0844bdc6e54169c55" candidate_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd714916c8de43f19d54f5ab3e533e69" is_pre_election: true
I20260808 04:45:18.524215 24850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82ee1212fca49d0844bdc6e54169c55" candidate_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619fa13ef56742af9bb40e2b01b08a5f" is_pre_election: true
I20260808 04:45:18.524247 24428 raft_consensus.cc:2454] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5de29c3a1a94f6c8f0924b081072aaa in term 0.
I20260808 04:45:18.524325 24850 raft_consensus.cc:2454] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5de29c3a1a94f6c8f0924b081072aaa in term 0.
I20260808 04:45:18.524619 24536 leader_election.cc:302] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd714916c8de43f19d54f5ab3e533e69, f5de29c3a1a94f6c8f0924b081072aaa; no voters:
I20260808 04:45:18.524822 24951 raft_consensus.cc:2781] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:18.524878 24951 raft_consensus.cc:482] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:18.524930 24951 raft_consensus.cc:3037] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.525800 24951 raft_consensus.cc:504] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.526162 24951 leader_election.cc:288] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [CANDIDATE]: Term 1 election: Requested vote from peers bd714916c8de43f19d54f5ab3e533e69 (127.22.2.129:38811), 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729)
I20260808 04:45:18.526477 24428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82ee1212fca49d0844bdc6e54169c55" candidate_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd714916c8de43f19d54f5ab3e533e69"
I20260808 04:45:18.526543 24850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82ee1212fca49d0844bdc6e54169c55" candidate_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619fa13ef56742af9bb40e2b01b08a5f"
I20260808 04:45:18.526592 24428 raft_consensus.cc:3037] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.526656 24850 raft_consensus.cc:3037] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:18.527473 24428 raft_consensus.cc:2454] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5de29c3a1a94f6c8f0924b081072aaa in term 1.
I20260808 04:45:18.527586 24850 raft_consensus.cc:2454] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5de29c3a1a94f6c8f0924b081072aaa in term 1.
I20260808 04:45:18.527842 24536 leader_election.cc:302] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd714916c8de43f19d54f5ab3e533e69, f5de29c3a1a94f6c8f0924b081072aaa; no voters:
I20260808 04:45:18.528020 24951 raft_consensus.cc:2781] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:18.528141 24951 raft_consensus.cc:686] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [term 1 LEADER]: Becoming Leader. State: Replica: f5de29c3a1a94f6c8f0924b081072aaa, State: Running, Role: LEADER
I20260808 04:45:18.528272 24951 consensus_queue.cc:237] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } }
I20260808 04:45:18.529058 24993 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: TxnStatusTablet state changed. Reason: New leader f5de29c3a1a94f6c8f0924b081072aaa. Latest consensus state: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.529189 24993 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:18.529546 25012 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 04:45:18.529752 25012 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 04:45:18.529932 24186 catalog_manager.cc:5495] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa reported cstate change: term changed from 0 to 1, leader changed from <none> to f5de29c3a1a94f6c8f0924b081072aaa (127.22.2.130). New cstate: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:18.545768 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.541413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41925 (local address 127.0.0.1:55618)
0808 04:45:18.541575 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.541592 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.541702 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.542043 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.542045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.542052 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:45:18.542147 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.542152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.543266 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.543269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.543876 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.543884 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.544459 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.544462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.544586 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.544643 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:45:18.545500 (+ 857us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:45:18.545515 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.545575 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.545648 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 04:45:18.545782 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.541500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.22.2.130:41925)
0808 04:45:18.541637 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.541644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.541659 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:18.541847 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.541850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.541912 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.542018 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:18.542260 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.543084 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.544000 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.544345 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.544453 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.544809 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:18.545388 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:18.545504 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.545594 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.545648 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 04:45:18.548722 24850 raft_consensus.cc:1261] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Refusing update from remote peer f5de29c3a1a94f6c8f0924b081072aaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:18.548910 24428 raft_consensus.cc:1261] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Refusing update from remote peer f5de29c3a1a94f6c8f0924b081072aaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:18.549273 24993 consensus_queue.cc:1035] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:18.549486 24951 consensus_queue.cc:1035] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:18.551227 24953 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: TxnStatusTablet state changed. Reason: New leader f5de29c3a1a94f6c8f0924b081072aaa. Latest consensus state: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.551335 24953 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:18.552764 24955 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: TxnStatusTablet state changed. Reason: New leader f5de29c3a1a94f6c8f0924b081072aaa. Latest consensus state: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.552892 24955 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:18.553941 24993 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.554044 24993 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:18.555061 24953 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.555159 24953 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P bd714916c8de43f19d54f5ab3e533e69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:18.558486 24951 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.558585 24951 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P f5de29c3a1a94f6c8f0924b081072aaa: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:18.559481 24955 tablet_replica.cc:439] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 } } peers { permanent_uuid: "f5de29c3a1a94f6c8f0924b081072aaa" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41925 } } peers { permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 } } }
I20260808 04:45:18.559587 24955 tablet_replica.cc:442] T d82ee1212fca49d0844bdc6e54169c55 P 619fa13ef56742af9bb40e2b01b08a5f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:18.563756 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.560244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.22.2.130:41925)
0808 04:45:18.560341 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.560345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.560360 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:18.560403 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.560405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.560467 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.560915 (+ 448us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:18.561016 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.561646 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.562810 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.563005 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.563078 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.563344 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:18.563407 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:18.563459 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.563571 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.563610 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 04:45:18.575385 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.571713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:43557 (local address 127.22.2.130:41925)
0808 04:45:18.571835 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.571838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.571853 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:18.571891 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.571893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.571932 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.572001 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:18.572330 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.572988 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.574284 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.574510 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.574599 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.574942 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:18.575008 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:18.575067 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.575189 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.575233 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":46}
I20260808 04:45:18.575448 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.571390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41925 (local address 127.22.2.130:43557)
0808 04:45:18.571704 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:18.571719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:18.571813 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:18.572029 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:18.572032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:18.572039 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:45:18.572219 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.572226 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.573096 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.573100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.574146 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:18.574157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.574626 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.574630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:18.574748 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.574806 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:45:18.575091 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:45:18.575097 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 04:45:18.575177 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 04:45:18.575257 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":18,"thread_start_us":98,"threads_started":1}
I20260808 04:45:18.583925 24850 raft_consensus.cc:1261] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Refusing update from remote peer f5de29c3a1a94f6c8f0924b081072aaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:18.583957 24428 raft_consensus.cc:1261] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Refusing update from remote peer f5de29c3a1a94f6c8f0924b081072aaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:18.584340 25022 consensus_queue.cc:1035] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd714916c8de43f19d54f5ab3e533e69" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:18.584501 24951 consensus_queue.cc:1035] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "619fa13ef56742af9bb40e2b01b08a5f" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:18.635927 24186 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:18.636094 24186 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:18.638269 24186 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76944cb3298e46b4bb27c7ed4d1712f7: Sending DeleteTablet for 3 replicas of tablet 5f14fdbb886f4a9ba96794dc0295ebdd
I20260808 04:45:18.639068 24819 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f14fdbb886f4a9ba96794dc0295ebdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:43796
I20260808 04:45:18.639052 24611 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f14fdbb886f4a9ba96794dc0295ebdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:55614
I20260808 04:45:18.639276 24981 tablet_replica.cc:330] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: stopping tablet replica
I20260808 04:45:18.639405 24981 raft_consensus.cc:2229] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:18.639755 24981 raft_consensus.cc:2258] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:18.639844 24397 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f14fdbb886f4a9ba96794dc0295ebdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:41450
I20260808 04:45:18.640054 24983 tablet_replica.cc:330] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: stopping tablet replica
I20260808 04:45:18.640168 24983 raft_consensus.cc:2229] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:18.640275 24983 raft_consensus.cc:2258] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:18.640421 24985 tablet_replica.cc:330] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: stopping tablet replica
I20260808 04:45:18.640569 24985 raft_consensus.cc:2229] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:18.640688 24985 raft_consensus.cc:2258] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:18.640714 24981 ts_tablet_manager.cc:1905] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:18.641546 24983 ts_tablet_manager.cc:1905] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:18.643283 24981 ts_tablet_manager.cc:1918] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:18.643378 24981 log.cc:1196] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/5f14fdbb886f4a9ba96794dc0295ebdd
I20260808 04:45:18.643698 24981 ts_tablet_manager.cc:1939] T 5f14fdbb886f4a9ba96794dc0295ebdd P f5de29c3a1a94f6c8f0924b081072aaa: Deleting consensus metadata
I20260808 04:45:18.644112 24983 ts_tablet_manager.cc:1918] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:18.644191 24983 log.cc:1196] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/5f14fdbb886f4a9ba96794dc0295ebdd
I20260808 04:45:18.644376 24168 catalog_manager.cc:4841] TS f5de29c3a1a94f6c8f0924b081072aaa (127.22.2.130:41925): tablet 5f14fdbb886f4a9ba96794dc0295ebdd (table test-table [id=93c6480b8c044884bfc93db16c72fe21]) successfully deleted
I20260808 04:45:18.644429 24983 ts_tablet_manager.cc:1939] T 5f14fdbb886f4a9ba96794dc0295ebdd P 619fa13ef56742af9bb40e2b01b08a5f: Deleting consensus metadata
I20260808 04:45:18.645109 24167 catalog_manager.cc:4841] TS 619fa13ef56742af9bb40e2b01b08a5f (127.22.2.131:46729): tablet 5f14fdbb886f4a9ba96794dc0295ebdd (table test-table [id=93c6480b8c044884bfc93db16c72fe21]) successfully deleted
I20260808 04:45:18.645246 24985 ts_tablet_manager.cc:1905] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:18.647740 24985 ts_tablet_manager.cc:1918] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:18.647801 24985 log.cc:1196] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/5f14fdbb886f4a9ba96794dc0295ebdd
I20260808 04:45:18.648018 24985 ts_tablet_manager.cc:1939] T 5f14fdbb886f4a9ba96794dc0295ebdd P bd714916c8de43f19d54f5ab3e533e69: Deleting consensus metadata
I20260808 04:45:18.648803 24168 catalog_manager.cc:4841] TS bd714916c8de43f19d54f5ab3e533e69 (127.22.2.129:38811): tablet 5f14fdbb886f4a9ba96794dc0295ebdd (table test-table [id=93c6480b8c044884bfc93db16c72fe21]) successfully deleted
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[24121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164318, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:18.657403 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.640243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41466 (local address 127.22.2.129:38811)
0808 04:45:18.640375 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.640380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.645085 (+ 4705us) server_negotiation.cc:407] Connection header received
0808 04:45:18.645265 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.645268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.645315 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.645364 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:18.646324 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.647106 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.648342 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.648718 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.648837 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.652349 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:18.652361 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:18.652386 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:18.652412 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:18.654734 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.655284 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.655313 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.655317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.655443 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.655776 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.655782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.655784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.656125 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:18.656353 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.657167 (+ 814us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.657287 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 04:45:18.658003 24478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41466
I20260808 04:45:18.667476 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:18.659857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.22.2.190:46041)
0808 04:45:18.659983 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:18.659990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:18.660051 (+ 61us) server_negotiation.cc:407] Connection header received
0808 04:45:18.660239 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:18.660243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:18.660296 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:18.660382 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:18.661142 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.661668 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.662466 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:18.662707 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:18.662818 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:18.664114 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:18.664117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:18.664120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:18.664161 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:18.665640 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.666080 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.666085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.666087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.666186 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:18.666507 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:18.666510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:18.666512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:18.666694 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:18.666834 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:18.667148 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 04:45:18.667271 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 04:45:18.667959 24186 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32808
I20260808 04:45:18.669410 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 24300
I20260808 04:45:18.683396 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 24506
I20260808 04:45:18.693284 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 24714
I20260808 04:45:18.703001 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 24142
2026-08-08T04:45:18Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3016 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[25035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[25035](info): set up 2 sockets
Aug 08 04:45:18 dist-test-slave-04bb krb5kdc[25035](info): commencing operation
krb5kdc: starting...
W20260808 04:45:19.949839 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:20 dist-test-slave-04bb krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:20.079952 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36981
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:36981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:20.200544 25054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:20.200848 25054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:20.200922 25054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:20.204957 25054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:20.205027 25054 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 04:45:20.205058 25054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:20.205099 25054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:20.205139 25054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:20.205174 25054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:20.210304 25054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:36981
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:20.211424 25054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:20.212450 25054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:20.220976 25061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:20.221014 25065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:20.220983 25067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:20.221379 25054 server_base.cc:1034] running on GCE node
I20260808 04:45:20.221959 25054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:20.223052 25054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:20.224213 25054 hybrid_clock.cc:648] HybridClock initialized: now 1786164320224184 us; error 41 us; skew 500 ppm
Aug 08 04:45:20 dist-test-slave-04bb krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:20.226976 25054 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:20.227777 25054 webserver.cc:460] Webserver started at http://127.22.2.190:36713/ using document root <none> and password file <none>
I20260808 04:45:20.228292 25054 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:20.228358 25054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:20.228518 25054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:20.230562 25054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "61bb630d7f724d0295d60cdbab38e675"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "99b7dcc99c54b84ae33d2625d12e1887"
server_key_iv: "cf3ff296b64e81aaebb1f0525c6c1924"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.231062 25054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "61bb630d7f724d0295d60cdbab38e675"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "99b7dcc99c54b84ae33d2625d12e1887"
server_key_iv: "cf3ff296b64e81aaebb1f0525c6c1924"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.234716 25054 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:20.237085 25079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:20.238319 25054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:20.238463 25054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "61bb630d7f724d0295d60cdbab38e675"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "99b7dcc99c54b84ae33d2625d12e1887"
server_key_iv: "cf3ff296b64e81aaebb1f0525c6c1924"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.238575 25054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:20.258286 25054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:20.261225 25054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:20.261435 25054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:20.269405 25054 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:36981
I20260808 04:45:20.269429 25176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:36981 every 8 connection(s)
I20260808 04:45:20.270298 25054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:20.273505 25177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:20.276412 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 25054
I20260808 04:45:20.276552 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:20.276859 22538 external_mini_cluster.cc:1428] Setting key b39df6e3b67e9260c9170c0ffb0432ad
I20260808 04:45:20.279495 25177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: Bootstrap starting.
I20260808 04:45:20.282048 25177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:20.282928 25177 log.cc:824] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:20.284972 25177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: No bootstrap required, opened a new log
Aug 08 04:45:20 dist-test-slave-04bb krb5kdc[25035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:20.288725 25177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bb630d7f724d0295d60cdbab38e675" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36981 } }
I20260808 04:45:20.289058 25177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:20.289139 25177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61bb630d7f724d0295d60cdbab38e675, State: Initialized, Role: FOLLOWER
I20260808 04:45:20.289644 25177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bb630d7f724d0295d60cdbab38e675" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36981 } }
I20260808 04:45:20.289814 25177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:20.289882 25177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:20.289979 25177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:20.290931 25177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bb630d7f724d0295d60cdbab38e675" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36981 } }
I20260808 04:45:20.291343 25177 leader_election.cc:302] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61bb630d7f724d0295d60cdbab38e675; no voters:
I20260808 04:45:20.291708 25177 leader_election.cc:288] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:20.291742 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.278224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.22.2.190:36981)
0808 04:45:20.278621 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:20.278637 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:20.278671 (+ 34us) server_negotiation.cc:407] Connection header received
0808 04:45:20.279257 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:20.279273 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:20.279651 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:20.279927 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:20.280834 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.281808 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.283039 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.283521 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.283701 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.286238 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:20.286244 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:20.286257 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:20.286282 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:20.288008 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.288480 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.288485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.288490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.288695 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.289052 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.289058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.289062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.289624 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:20.289850 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:20.290152 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 04:45:20.290660 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":126,"threads_started":1}
I20260808 04:45:20.291831 25187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:20.292089 25187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [term 1 LEADER]: Becoming Leader. State: Replica: 61bb630d7f724d0295d60cdbab38e675, State: Running, Role: LEADER
I20260808 04:45:20.292418 25187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bb630d7f724d0295d60cdbab38e675" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36981 } }
I20260808 04:45:20.293208 25177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:20.293679 25187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61bb630d7f724d0295d60cdbab38e675. Latest consensus state: current_term: 1 leader_uuid: "61bb630d7f724d0295d60cdbab38e675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bb630d7f724d0295d60cdbab38e675" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36981 } } }
I20260808 04:45:20.293782 25187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:20.295166 25190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61bb630d7f724d0295d60cdbab38e675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bb630d7f724d0295d60cdbab38e675" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36981 } } }
I20260808 04:45:20.295272 25190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675 [sys.catalog]: This master's current role is: LEADER
W20260808 04:45:20.298282 25211 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:45:20.298354 25211 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:45:20.298429 25194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:20.299263 25194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:20.305336 25194 catalog_manager.cc:1347] Generated new cluster ID: 83f7abe026f44c17aa0701306aa3f285
I20260808 04:45:20.305418 25194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:20.309834 25194 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:20.311056 25194 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:20.316599 25194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: Generated new TSK 0
I20260808 04:45:20.317198 25194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:20.357115 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36981
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:20.483115 25217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:20.483366 25217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:20.483409 25217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:20.487355 25217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:20.487411 25217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:20.487450 25217 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 04:45:20.487464 25217 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:20.487495 25217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:20.492498 25217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:36981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:20.493654 25217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:20.494673 25217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:20.504381 25228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:20.504400 25232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:20.504387 25229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:20.504717 25217 server_base.cc:1034] running on GCE node
I20260808 04:45:20.505214 25217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:20.505913 25217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:20.507115 25217 hybrid_clock.cc:648] HybridClock initialized: now 1786164320507095 us; error 30 us; skew 500 ppm
Aug 08 04:45:20 dist-test-slave-04bb krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:20.510080 25217 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:20.510977 25217 webserver.cc:460] Webserver started at http://127.22.2.129:36525/ using document root <none> and password file <none>
I20260808 04:45:20.511597 25217 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:20.511734 25217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:20.511971 25217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:20.514289 25217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "353187d5c79376486250fc966d354c5f"
server_key_iv: "ce5e0929c2d59a85fdd93230dc0e92be"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.514864 25217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "353187d5c79376486250fc966d354c5f"
server_key_iv: "ce5e0929c2d59a85fdd93230dc0e92be"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.518653 25217 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 04:45:20.521111 25244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:20.522295 25217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:20.522435 25217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "353187d5c79376486250fc966d354c5f"
server_key_iv: "ce5e0929c2d59a85fdd93230dc0e92be"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.522549 25217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:20.545369 25217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:20.548877 25217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:20.549094 25217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:20.550571 25217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:20.550647 25217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:20.550732 25217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:20.550787 25217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:45:20 dist-test-slave-04bb krb5kdc[25035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:20.563257 25217 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:35075
I20260808 04:45:20.563690 25407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:35075 every 8 connection(s)
I20260808 04:45:20.564592 25217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:20.566253 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.551534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48085 (local address 127.22.2.190:36981)
0808 04:45:20.551698 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:20.551702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:20.552915 (+ 1213us) server_negotiation.cc:407] Connection header received
0808 04:45:20.554241 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:20.554248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:20.554321 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:20.554424 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:20.555886 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.556834 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.558365 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.558649 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.558796 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.561889 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:20.561896 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:20.561902 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:20.561938 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:20.563690 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.564385 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.564389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.564390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.564509 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.564950 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.564953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.564954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.565250 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:20.565351 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:20.565956 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 04:45:20.566118 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:45:20.566995 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.552076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36981 (local address 127.22.2.129:48085)
0808 04:45:20.552746 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:20.552796 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:20.553952 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:20.554675 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:20.554686 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:20.555101 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:20.555727 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:20.555744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.556998 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.557004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:20.558207 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:20.558219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.558865 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.558871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:20.559108 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.559752 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:20.559772 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:20.561691 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:20.563827 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:20.563833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:20.563849 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:20.564262 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:20.564631 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:20.564639 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:20.564643 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:20.564837 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:20.565410 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:20.565417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:20.565709 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 04:45:20.565872 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 04:45:20.566116 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":147,"threads_started":1}
I20260808 04:45:20.568365 25411 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36981
I20260808 04:45:20.568507 25411 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:20.569124 25411 heartbeater.cc:511] Master 127.22.2.190:36981 requested a full tablet report, sending...
I20260808 04:45:20.570782 25106 ts_manager.cc:194] Registered new tserver with Master: bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129:35075)
I20260808 04:45:20.572296 25106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:48085
I20260808 04:45:20.574563 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 25217
I20260808 04:45:20.574702 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:20.575079 22538 external_mini_cluster.cc:1428] Setting key 1f1badffedb95c62487ad6bc471f6675
I20260808 04:45:20.578220 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.570771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:52001 (local address 127.22.2.190:36981)
0808 04:45:20.570897 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:20.570900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:20.571162 (+ 262us) server_negotiation.cc:407] Connection header received
0808 04:45:20.571244 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:20.571247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:20.571293 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:20.571384 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:20.571805 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.572557 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.573456 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.573664 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.573739 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.574930 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:20.574934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:20.574937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:20.574965 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:20.576616 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.577044 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.577047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.577049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.577136 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.577420 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.577424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.577426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.577596 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:20.577711 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:20.577962 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 04:45:20.578070 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 04:45:20.578253 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.570683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36981 (local address 127.22.2.129:52001)
0808 04:45:20.571076 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:20.571088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:20.571171 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:20.571418 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:20.571420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:20.571580 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:20.571693 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:20.571698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.572695 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.572698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:20.573324 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:20.573335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.573776 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.573780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:20.573868 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.574346 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:20.574360 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:20.574787 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:20.576744 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:20.576746 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:20.576748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:20.576932 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:20.577235 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:20.577238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:20.577240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:20.577323 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:20.577700 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:20.577703 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:20.577796 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 04:45:20.577906 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 04:45:20.578049 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:20.604162 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36981
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:20.718918 25426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:20.719163 25426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:20.719277 25426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:20.723232 25426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:20.723285 25426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:20.723325 25426 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 04:45:20.723338 25426 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:20.723369 25426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:20.728629 25426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:36981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:20.729655 25426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:20.730628 25426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:20.740203 25436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:20.740203 25438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:20.740338 25426 server_base.cc:1034] running on GCE node
W20260808 04:45:20.740303 25435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:20.740931 25426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:20.741609 25426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:20.742846 25426 hybrid_clock.cc:648] HybridClock initialized: now 1786164320742799 us; error 37 us; skew 500 ppm
Aug 08 04:45:20 dist-test-slave-04bb krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:20.748991 25426 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:20.749873 25426 webserver.cc:460] Webserver started at http://127.22.2.130:36921/ using document root <none> and password file <none>
I20260808 04:45:20.750453 25426 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:20.750543 25426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:20.750792 25426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:20.752915 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "382e1f31e34441b28d28e343da52bbc8"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "a4ca91e1d5e36333d951b62f5f0815e0"
server_key_iv: "95c58a206398459af0c59b48251d630a"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.753479 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "382e1f31e34441b28d28e343da52bbc8"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "a4ca91e1d5e36333d951b62f5f0815e0"
server_key_iv: "95c58a206398459af0c59b48251d630a"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.757228 25426 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:20.759608 25450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:20.761016 25426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:20.761161 25426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "382e1f31e34441b28d28e343da52bbc8"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "a4ca91e1d5e36333d951b62f5f0815e0"
server_key_iv: "95c58a206398459af0c59b48251d630a"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.761286 25426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:20.773355 25426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:20.776016 25426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:20.776225 25426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:20.777629 25426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:20.777705 25426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:20.777771 25426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:20.777817 25426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:45:20 dist-test-slave-04bb krb5kdc[25035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:20.789000 25426 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:33857
I20260808 04:45:20.789892 25627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:33857 every 8 connection(s)
I20260808 04:45:20.790292 25426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:20.792168 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.778504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:44677 (local address 127.22.2.190:36981)
0808 04:45:20.778639 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:20.778643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:20.779663 (+ 1020us) server_negotiation.cc:407] Connection header received
0808 04:45:20.780754 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:20.780759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:20.780824 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:20.780890 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:20.782359 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.782950 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.784453 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.784739 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.784856 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.787880 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:20.787884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:20.787887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:20.787922 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:20.789658 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.790356 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.790361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.790365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.790494 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.790984 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.790987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.790989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.791237 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:20.791350 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:20.791859 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 04:45:20.791995 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:45:20.792995 25464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.779021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36981 (local address 127.22.2.130:44677)
0808 04:45:20.779507 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:20.779547 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:20.780460 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:20.781348 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:20.781358 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:20.781758 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:20.782206 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:20.782219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.783087 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.783091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:20.784308 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:20.784319 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.784908 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.784912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:20.785086 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.785738 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:20.785756 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:20.787694 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:20.789816 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:20.789822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:20.789833 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:20.790207 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:20.790613 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:20.790619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:20.790622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:20.790861 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:20.791345 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:20.791352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:20.791604 (+ 252us) client_negotiation.cc:769] Sending connection context
0808 04:45:20.791802 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 04:45:20.792125 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":143,"threads_started":1}
I20260808 04:45:20.794342 25628 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36981
I20260808 04:45:20.794543 25628 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:20.795091 25628 heartbeater.cc:511] Master 127.22.2.190:36981 requested a full tablet report, sending...
I20260808 04:45:20.796197 25108 ts_manager.cc:194] Registered new tserver with Master: 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857)
I20260808 04:45:20.796984 25108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:44677
I20260808 04:45:20.800623 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 25426
I20260808 04:45:20.800758 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:20.801050 22538 external_mini_cluster.cc:1428] Setting key 8ee0bbcbffc94919f37b9c0575223fca
I20260808 04:45:20.804160 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.796708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:34161 (local address 127.22.2.190:36981)
0808 04:45:20.796827 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:20.796831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:20.797022 (+ 191us) server_negotiation.cc:407] Connection header received
0808 04:45:20.797200 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:20.797203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:20.797244 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:20.797346 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:20.797842 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.798354 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.799355 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.799563 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.799647 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.800872 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:20.800876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:20.800879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:20.800904 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:20.802464 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.802899 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.802903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.802904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.802990 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:20.803276 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:20.803280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:20.803281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:20.803502 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:20.803616 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:20.803899 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 04:45:20.804006 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 04:45:20.804176 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:20.796668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36981 (local address 127.22.2.130:34161)
0808 04:45:20.796950 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:20.796965 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:20.797070 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:20.797380 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:20.797383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:20.797596 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:20.797723 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:20.797731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.798489 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.798492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:20.799231 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:20.799240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:20.799661 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:20.799663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:20.799729 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:20.800276 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:20.800293 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:20.800679 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:20.802583 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:20.802586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:20.802588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:20.802788 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:20.803089 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:20.803092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:20.803094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:20.803182 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:20.803608 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:20.803611 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:20.803710 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 04:45:20.803837 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 04:45:20.803981 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:20.832916 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36981
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:20.950425 25647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:20.950661 25647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:20.950706 25647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:20.954636 25647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:20.954695 25647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:20.954734 25647 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 04:45:20.954748 25647 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:20.954779 25647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:20.959800 25647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:36981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:20.960773 25647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:20.961714 25647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:20.971781 25661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:20.971649 25655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:20.971653 25656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:20.972143 25647 server_base.cc:1034] running on GCE node
I20260808 04:45:20.972505 25647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:20.973173 25647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:20.974367 25647 hybrid_clock.cc:648] HybridClock initialized: now 1786164320974351 us; error 25 us; skew 500 ppm
Aug 08 04:45:20 dist-test-slave-04bb krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:20.977512 25647 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:20.978363 25647 webserver.cc:460] Webserver started at http://127.22.2.131:34779/ using document root <none> and password file <none>
I20260808 04:45:20.978924 25647 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:20.978986 25647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:20.979241 25647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:20.981354 25647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "959ea4a9a76f4ea9952cdcba9588f94e"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "d9f921ecbc038c367b57bb304352ac25"
server_key_iv: "ea69df400af1fa7e6870011a036b4d2c"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.981922 25647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "959ea4a9a76f4ea9952cdcba9588f94e"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "d9f921ecbc038c367b57bb304352ac25"
server_key_iv: "ea69df400af1fa7e6870011a036b4d2c"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.985628 25647 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:45:20.987928 25673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:20.989190 25647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:20.989326 25647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "959ea4a9a76f4ea9952cdcba9588f94e"
format_stamp: "Formatted at 2026-08-08 04:45:20 on dist-test-slave-04bb"
server_key: "d9f921ecbc038c367b57bb304352ac25"
server_key_iv: "ea69df400af1fa7e6870011a036b4d2c"
server_key_version: "encryptionkey@0"
I20260808 04:45:20.989439 25647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:21.007288 25647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:21.010492 25647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:21.010699 25647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:21.012121 25647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:21.012214 25647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:21.012306 25647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:21.012357 25647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:45:21 dist-test-slave-04bb krb5kdc[25035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:21.023835 25647 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:42251
I20260808 04:45:21.024330 25854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:42251 every 8 connection(s)
I20260808 04:45:21.024945 25647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:21.027675 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.013322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:53243 (local address 127.22.2.190:36981)
0808 04:45:21.013514 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.013521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.014502 (+ 981us) server_negotiation.cc:407] Connection header received
0808 04:45:21.015859 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.015864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.015938 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.016076 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:21.017553 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.018272 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.020162 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.020608 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.020745 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.023914 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:21.023918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:21.023920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:21.023968 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:21.025736 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:21.026327 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:21.026330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:21.026332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:21.026424 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:21.026739 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:21.026742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:21.026743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:21.026926 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:21.027004 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.027449 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.027553 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 04:45:21.028599 25687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.013721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36981 (local address 127.22.2.131:53243)
0808 04:45:21.014321 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.014383 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.015539 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.016277 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.016284 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.016715 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:21.017382 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.017411 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.018422 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.018428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.020001 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.020017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.020923 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.020926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.021080 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.021812 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:21.021836 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:21.023696 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:21.025883 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:21.025887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:21.025899 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:21.026228 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:21.026517 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:21.026519 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:21.026520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:21.026652 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:21.027015 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:21.027020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:21.027241 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.027413 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.027660 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":146,"threads_started":1}
I20260808 04:45:21.029803 25855 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36981
I20260808 04:45:21.029975 25855 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:21.030067 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 25647
I20260808 04:45:21.030191 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:21.030480 22538 external_mini_cluster.cc:1428] Setting key f3d30bc69629a61c517d911a6978860f
I20260808 04:45:21.030627 25855 heartbeater.cc:511] Master 127.22.2.190:36981 requested a full tablet report, sending...
I20260808 04:45:21.031834 25108 ts_manager.cc:194] Registered new tserver with Master: 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:21.032500 25108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:53243
I20260808 04:45:21.033030 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:45:21.041827 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.032798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:45095 (local address 127.22.2.190:36981)
0808 04:45:21.032932 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.032937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.033281 (+ 344us) server_negotiation.cc:407] Connection header received
0808 04:45:21.033408 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.033410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.033448 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.033512 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:21.034154 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.034628 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.036008 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.036392 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.036510 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.037931 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:21.037937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:21.037943 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:21.037975 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:21.039759 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:21.040369 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:21.040371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:21.040372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:21.040450 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:21.040839 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:21.040842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:21.040843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:21.041004 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:21.041082 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.041497 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.041623 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:45:21.041976 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.032706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36981 (local address 127.22.2.131:45095)
0808 04:45:21.033179 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.033197 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.033313 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.033568 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.033573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.033843 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:21.034010 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.034023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.034766 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.034773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.035863 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.035876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.036562 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.036567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.036675 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.037272 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:21.037314 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:21.037781 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:21.039897 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:21.039900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:21.039902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:21.040238 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:21.040586 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:21.040592 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:21.040596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:21.040723 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:21.041119 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:21.041126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:21.041265 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.041423 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.041628 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":229,"threads_started":1}
Aug 08 04:45:21 dist-test-slave-04bb krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:45:21 dist-test-slave-04bb krb5kdc[25035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:21.062594 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.054412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.22.2.190:36981)
0808 04:45:21.054540 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.054543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.054555 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:45:21.054588 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.054590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.054626 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.054716 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:21.055421 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.055934 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.056901 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.057121 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.057231 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.059609 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:21.059614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:21.059616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:21.059639 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:21.060926 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:21.061353 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:21.061357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:21.061359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:21.061441 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:21.061728 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:21.061731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:21.061733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:21.061912 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:21.062054 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.062293 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.062431 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:21.065564 25108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:21.068740 25108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:21.083571 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.077554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35075 (local address 127.0.0.1:39414)
0808 04:45:21.077969 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.078024 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.078155 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.078969 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.078975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.078999 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:21.079132 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.079138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.080314 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.080318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.081870 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.081882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.082960 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.082964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.083076 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.083123 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.083248 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.083402 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260808 04:45:21.083752 25886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.078735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.22.2.130:33857)
0808 04:45:21.079189 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.079203 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.079229 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:45:21.079308 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.079312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.079493 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.079657 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:21.079882 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.081006 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.082505 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.083157 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.083269 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.083441 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.083506 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.083598 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":113,"threads_started":1}
I20260808 04:45:21.084066 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.077793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:33857 (local address 127.0.0.1:51268)
0808 04:45:21.078453 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.078467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.078549 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.079644 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.079646 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.079666 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:21.079759 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.079764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.081169 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.081176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.082380 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.082390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.083282 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.083286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.083390 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.083416 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.083484 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.083564 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":31,"thread_start_us":96,"threads_started":1}
I20260808 04:45:21.084481 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.077702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.22.2.129:35075)
0808 04:45:21.078125 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.078132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.078157 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:45:21.078367 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.078374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.078599 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.078757 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:21.079274 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.080196 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.082012 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.082825 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.082936 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.083075 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.084240 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.084349 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":101,"threads_started":1}
I20260808 04:45:21.085294 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.078276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:42251 (local address 127.0.0.1:38556)
0808 04:45:21.079144 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.079157 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.079236 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.079882 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.079885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.079904 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:21.080008 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.080014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.081247 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.081251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.083437 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.083449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.084898 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.084901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.084997 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.085020 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.085095 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.085163 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":27,"thread_start_us":90,"threads_started":1}
I20260808 04:45:21.085726 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.078749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.22.2.131:42251)
0808 04:45:21.079334 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.079345 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.079362 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:45:21.079424 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.079428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.079586 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.079712 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:21.080139 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.081117 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.083571 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.084465 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.084685 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.084842 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.085467 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.085576 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":122,"threads_started":1}
I20260808 04:45:21.087538 25528 tablet_service.cc:1467] Processing CreateTablet for tablet ad1a9f4b1c53434798ef9752e837c0dd (DEFAULT_TABLE table=test-table [id=16c9798128614d90b14862bd92af25df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:21.088778 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1a9f4b1c53434798ef9752e837c0dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.088382 25753 tablet_service.cc:1467] Processing CreateTablet for tablet ad1a9f4b1c53434798ef9752e837c0dd (DEFAULT_TABLE table=test-table [id=16c9798128614d90b14862bd92af25df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:21.089478 25753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1a9f4b1c53434798ef9752e837c0dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.089951 25321 tablet_service.cc:1467] Processing CreateTablet for tablet ad1a9f4b1c53434798ef9752e837c0dd (DEFAULT_TABLE table=test-table [id=16c9798128614d90b14862bd92af25df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:21.090797 25321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1a9f4b1c53434798ef9752e837c0dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.097595 25888 tablet_bootstrap.cc:492] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Bootstrap starting.
I20260808 04:45:21.098711 25890 tablet_bootstrap.cc:492] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: Bootstrap starting.
I20260808 04:45:21.099046 25889 tablet_bootstrap.cc:492] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: Bootstrap starting.
I20260808 04:45:21.099812 25888 tablet_bootstrap.cc:654] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.100622 25888 log.cc:824] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:21.101148 25889 tablet_bootstrap.cc:654] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.101140 25890 tablet_bootstrap.cc:654] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.102093 25889 log.cc:824] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:21.102094 25890 log.cc:824] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:21.103096 25888 tablet_bootstrap.cc:492] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: No bootstrap required, opened a new log
I20260808 04:45:21.103324 25888 ts_tablet_manager.cc:1397] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:45:21.104221 25890 tablet_bootstrap.cc:492] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: No bootstrap required, opened a new log
I20260808 04:45:21.104427 25890 ts_tablet_manager.cc:1397] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:45:21.104682 25889 tablet_bootstrap.cc:492] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: No bootstrap required, opened a new log
I20260808 04:45:21.104874 25889 ts_tablet_manager.cc:1397] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:45:21.106974 25888 raft_consensus.cc:348] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.107260 25888 raft_consensus.cc:374] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.107331 25888 raft_consensus.cc:729] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.107849 25888 consensus_queue.cc:260] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.108572 25411 heartbeater.cc:503] Master 127.22.2.190:36981 was elected leader, sending a full tablet report...
I20260808 04:45:21.108933 25888 ts_tablet_manager.cc:1428] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 04:45:21.108808 25890 raft_consensus.cc:348] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.109081 25890 raft_consensus.cc:374] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.109187 25890 raft_consensus.cc:729] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ea4a9a76f4ea9952cdcba9588f94e, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.109666 25890 consensus_queue.cc:260] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.110416 25889 raft_consensus.cc:348] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.110682 25889 raft_consensus.cc:374] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.110740 25890 ts_tablet_manager.cc:1428] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 04:45:21.110752 25889 raft_consensus.cc:729] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 382e1f31e34441b28d28e343da52bbc8, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.111102 25855 heartbeater.cc:503] Master 127.22.2.190:36981 was elected leader, sending a full tablet report...
I20260808 04:45:21.111224 25889 consensus_queue.cc:260] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.111935 25628 heartbeater.cc:503] Master 127.22.2.190:36981 was elected leader, sending a full tablet report...
I20260808 04:45:21.112285 25889 ts_tablet_manager.cc:1428] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 04:45:21.162282 25896 raft_consensus.cc:482] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:21.162496 25896 raft_consensus.cc:504] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.163695 25896 leader_election.cc:288] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:21.168135 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.163900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:33857 (local address 127.22.2.129:36673)
0808 04:45:21.164344 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.164364 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.164502 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.164977 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.164979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.164995 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:21.165151 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.165158 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.166041 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.166045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.166939 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.166949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.167613 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.167617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.167717 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.167746 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.167885 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.167962 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":153,"threads_started":1}
I20260808 04:45:21.168156 25886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.164078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:36673 (local address 127.22.2.130:33857)
0808 04:45:21.164254 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.164260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.164477 (+ 217us) server_negotiation.cc:407] Connection header received
0808 04:45:21.164700 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.164705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.164768 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.164828 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:21.165336 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.165919 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.167077 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.167484 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.167617 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.167672 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.167914 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.167999 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 04:45:21.168356 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.164574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56367 (local address 127.22.2.131:42251)
0808 04:45:21.164709 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.164714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.164801 (+ 87us) server_negotiation.cc:407] Connection header received
0808 04:45:21.164968 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.164972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.165029 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.165116 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:21.165471 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.165993 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.167144 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.167742 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.167877 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.167930 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.168146 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.168223 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:45:21.168368 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.164381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:42251 (local address 127.22.2.129:56367)
0808 04:45:21.164698 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.164710 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.164847 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.165167 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.165170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.165243 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:21.165348 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.165353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.166107 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.166114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.167010 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.167020 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.167907 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.167911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.168012 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.168033 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.168109 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.168181 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":56,"thread_start_us":114,"threads_started":1}
I20260808 04:45:21.169319 25756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1a9f4b1c53434798ef9752e837c0dd" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" is_pre_election: true
I20260808 04:45:21.169329 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1a9f4b1c53434798ef9752e837c0dd" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8" is_pre_election: true
I20260808 04:45:21.169653 25557 raft_consensus.cc:2454] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:21.169653 25756 raft_consensus.cc:2454] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:21.170145 25250 leader_election.cc:302] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 382e1f31e34441b28d28e343da52bbc8, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:21.170396 25896 raft_consensus.cc:2781] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:21.170481 25896 raft_consensus.cc:482] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:21.170514 25896 raft_consensus.cc:3037] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.171610 25896 raft_consensus.cc:504] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.171968 25896 leader_election.cc:288] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Requested vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:21.172392 25756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1a9f4b1c53434798ef9752e837c0dd" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e"
I20260808 04:45:21.172392 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1a9f4b1c53434798ef9752e837c0dd" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8"
I20260808 04:45:21.172544 25756 raft_consensus.cc:3037] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.172544 25557 raft_consensus.cc:3037] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.173846 25557 raft_consensus.cc:2454] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:21.173861 25756 raft_consensus.cc:2454] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:21.174161 25250 leader_election.cc:302] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959ea4a9a76f4ea9952cdcba9588f94e, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:21.174357 25896 raft_consensus.cc:2781] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:21.174573 25896 raft_consensus.cc:686] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 LEADER]: Becoming Leader. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Running, Role: LEADER
I20260808 04:45:21.174924 25896 consensus_queue.cc:237] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.178402 25106 catalog_manager.cc:5495] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } health_report { overall_health: UNKNOWN } } }
Aug 08 04:45:21 dist-test-slave-04bb krb5kdc[25035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164320, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:21.231629 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.222832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.22.2.190:36981)
0808 04:45:21.222997 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.223001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.223161 (+ 160us) server_negotiation.cc:407] Connection header received
0808 04:45:21.223295 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.223297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.223341 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.223425 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:21.224144 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.224762 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.225714 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.225935 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.226031 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.228377 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:21.228380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:21.228382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:21.228404 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:21.229946 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:21.230464 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:21.230466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:21.230468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:21.230585 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:21.230843 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:21.230845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:21.230846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:21.231020 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:21.231098 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.231396 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.231490 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 04:45:21.231635 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.222759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36981 (local address 127.0.0.1:56092)
0808 04:45:21.223051 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.223068 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.223202 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.223473 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.223475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.223915 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:21.224018 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.224025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.224867 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.224869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.225553 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.225563 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.226049 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.226051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.226116 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.226776 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:21.226788 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:21.228233 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:21.230052 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:21.230061 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:21.230066 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:21.230323 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:21.230671 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:21.230674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:21.230675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:21.230757 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:21.231115 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:21.231120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:21.231224 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.231329 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.231441 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
I20260808 04:45:21.235744 25104 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:56092:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 04:45:21.236492 25104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:21.241523 25321 tablet_service.cc:1467] Processing CreateTablet for tablet 29cbbadfce9e400b91a2da38fdae2e38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=475ca65695444edc9be4f77ec1d54164]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:21.241803 25753 tablet_service.cc:1467] Processing CreateTablet for tablet 29cbbadfce9e400b91a2da38fdae2e38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=475ca65695444edc9be4f77ec1d54164]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:21.241850 25528 tablet_service.cc:1467] Processing CreateTablet for tablet 29cbbadfce9e400b91a2da38fdae2e38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=475ca65695444edc9be4f77ec1d54164]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:21.242377 25321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cbbadfce9e400b91a2da38fdae2e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.242416 25753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cbbadfce9e400b91a2da38fdae2e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.242592 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cbbadfce9e400b91a2da38fdae2e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.245082 25889 tablet_bootstrap.cc:492] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: Bootstrap starting.
I20260808 04:45:21.245442 25888 tablet_bootstrap.cc:492] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Bootstrap starting.
I20260808 04:45:21.245774 25890 tablet_bootstrap.cc:492] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: Bootstrap starting.
I20260808 04:45:21.246194 25889 tablet_bootstrap.cc:654] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.246490 25888 tablet_bootstrap.cc:654] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.246846 25890 tablet_bootstrap.cc:654] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.248320 25890 tablet_bootstrap.cc:492] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: No bootstrap required, opened a new log
I20260808 04:45:21.248420 25890 ts_tablet_manager.cc:1397] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:21.248726 25889 tablet_bootstrap.cc:492] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: No bootstrap required, opened a new log
I20260808 04:45:21.248818 25889 ts_tablet_manager.cc:1397] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 04:45:21.248988 25890 raft_consensus.cc:348] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.249155 25890 raft_consensus.cc:374] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.249195 25890 raft_consensus.cc:729] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ea4a9a76f4ea9952cdcba9588f94e, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.249383 25888 tablet_bootstrap.cc:492] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: No bootstrap required, opened a new log
I20260808 04:45:21.249328 25890 consensus_queue.cc:260] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.249356 25889 raft_consensus.cc:348] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.249454 25889 raft_consensus.cc:374] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.249488 25888 ts_tablet_manager.cc:1397] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 04:45:21.249496 25889 raft_consensus.cc:729] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 382e1f31e34441b28d28e343da52bbc8, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.249642 25890 ts_tablet_manager.cc:1428] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:45:21.249610 25889 consensus_queue.cc:260] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.249665 25900 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.249818 25900 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:21.249847 25889 ts_tablet_manager.cc:1428] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:21.249948 25888 raft_consensus.cc:348] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.250065 25888 raft_consensus.cc:374] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.250097 25888 raft_consensus.cc:729] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.250108 25903 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.250236 25903 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:21.250231 25888 consensus_queue.cc:260] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.250532 25888 ts_tablet_manager.cc:1428] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:21.250509 25896 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.250649 25896 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 04:45:21.276068 25857 tablet.cc:2369] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:45:21.291853 25630 tablet.cc:2369] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:45:21.316161 25412 tablet.cc:2369] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:21.332893 25896 raft_consensus.cc:482] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:21.333076 25896 raft_consensus.cc:504] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.333560 25896 leader_election.cc:288] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:21.334141 25756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29cbbadfce9e400b91a2da38fdae2e38" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" is_pre_election: true
I20260808 04:45:21.334177 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29cbbadfce9e400b91a2da38fdae2e38" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8" is_pre_election: true
I20260808 04:45:21.334282 25756 raft_consensus.cc:2454] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:21.334380 25557 raft_consensus.cc:2454] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:21.334699 25250 leader_election.cc:302] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959ea4a9a76f4ea9952cdcba9588f94e, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:21.334939 25896 raft_consensus.cc:2781] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:21.335017 25896 raft_consensus.cc:482] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:21.335067 25896 raft_consensus.cc:3037] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.336071 25896 raft_consensus.cc:504] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.336397 25896 leader_election.cc:288] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Requested vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:21.336817 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29cbbadfce9e400b91a2da38fdae2e38" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8"
I20260808 04:45:21.336858 25756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29cbbadfce9e400b91a2da38fdae2e38" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e"
I20260808 04:45:21.336972 25557 raft_consensus.cc:3037] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.336992 25756 raft_consensus.cc:3037] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.337896 25756 raft_consensus.cc:2454] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:21.338101 25557 raft_consensus.cc:2454] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:21.338248 25250 leader_election.cc:302] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959ea4a9a76f4ea9952cdcba9588f94e, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:21.338474 25896 raft_consensus.cc:2781] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:21.338609 25896 raft_consensus.cc:686] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 LEADER]: Becoming Leader. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Running, Role: LEADER
I20260808 04:45:21.338765 25896 consensus_queue.cc:237] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.339538 25906 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: TxnStatusTablet state changed. Reason: New leader bcbb3f6f9dfe44f98ff3b71ef690bbd2. Latest consensus state: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.339666 25906 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:21.340034 25929 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 04:45:21.340312 25929 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 04:45:21.340838 25108 catalog_manager.cc:5495] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:21.354864 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.350632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.22.2.129:35075)
0808 04:45:21.350775 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.350779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.350791 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:45:21.350852 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.350855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.350905 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.350992 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:21.351333 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.352022 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.353355 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.353593 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.353730 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.354035 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:21.354451 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:21.354554 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.354689 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.354745 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 04:45:21.354887 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.350168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35075 (local address 127.0.0.1:39426)
0808 04:45:21.350430 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.350446 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.350553 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.351049 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.351053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.351062 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:45:21.351181 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.351189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.352195 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.352200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.353206 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.353218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.353722 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.353726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.353829 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.353885 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:45:21.354566 (+ 681us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:45:21.354582 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.354659 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.354749 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260808 04:45:21.360577 25557 raft_consensus.cc:1261] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:21.360584 25756 raft_consensus.cc:1261] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:21.361341 25906 consensus_queue.cc:1035] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:21.361579 25896 consensus_queue.cc:1035] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:21.365047 25900 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: TxnStatusTablet state changed. Reason: New leader bcbb3f6f9dfe44f98ff3b71ef690bbd2. Latest consensus state: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.365197 25900 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:21.365411 25903 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: TxnStatusTablet state changed. Reason: New leader bcbb3f6f9dfe44f98ff3b71ef690bbd2. Latest consensus state: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.365520 25903 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:21.368698 25906 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.368813 25906 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:21.368935 25934 mvcc.cc:205] Tried to move back new op lower bound from 7316129060285837312 to 7316129060205158400. Current Snapshot: MvccSnapshot[applied={T|T < 7316129060285837312}]
I20260808 04:45:21.369405 25903 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.369517 25903 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P 382e1f31e34441b28d28e343da52bbc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:21.370340 25935 mvcc.cc:205] Tried to move back new op lower bound from 7316129060285837312 to 7316129060205158400. Current Snapshot: MvccSnapshot[applied={T|T < 7316129060285837312}]
I20260808 04:45:21.373183 25896 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.373302 25896 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:21.374488 25900 tablet_replica.cc:439] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } } }
I20260808 04:45:21.374611 25900 tablet_replica.cc:442] T 29cbbadfce9e400b91a2da38fdae2e38 P 959ea4a9a76f4ea9952cdcba9588f94e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:21.375005 25937 mvcc.cc:205] Tried to move back new op lower bound from 7316129060285837312 to 7316129060205158400. Current Snapshot: MvccSnapshot[applied={T|T < 7316129060285837312}]
I20260808 04:45:21.384632 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.381259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39428 (local address 127.22.2.129:35075)
0808 04:45:21.381371 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.381375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.381440 (+ 65us) server_negotiation.cc:407] Connection header received
0808 04:45:21.381608 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.381613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.381658 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.381736 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:21.382003 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.382577 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.383491 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.383752 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.383860 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.384169 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:21.384245 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:21.384328 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.384423 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.384473 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 04:45:21.396672 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.392422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:46569 (local address 127.22.2.129:35075)
0808 04:45:21.392577 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:21.392581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:21.392870 (+ 289us) server_negotiation.cc:407] Connection header received
0808 04:45:21.392987 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:21.392989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:21.393030 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:21.393091 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:21.393387 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.394059 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.395481 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.395753 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.395835 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.396201 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:21.396267 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:21.396320 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:21.396454 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 04:45:21.396496 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:45:21.396891 25950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:21.392381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35075 (local address 127.22.2.129:46569)
0808 04:45:21.392775 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:21.392790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:21.392896 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:21.393125 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:21.393128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:21.393135 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:45:21.393259 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.393265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.394161 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.394164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.395309 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:21.395322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:21.395866 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:21.395870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:21.395981 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:21.396035 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:45:21.396356 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:45:21.396363 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 04:45:21.396431 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 04:45:21.396509 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":139,"threads_started":1}
I20260808 04:45:21.403975 25756 raft_consensus.cc:1261] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:21.404035 25557 raft_consensus.cc:1261] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:21.404505 25896 consensus_queue.cc:1035] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:21.404685 25942 consensus_queue.cc:1035] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:21.448376 25104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:21.448588 25104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:21.450980 25104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: Sending DeleteTablet for 3 replicas of tablet ad1a9f4b1c53434798ef9752e837c0dd
I20260808 04:45:21.451673 25319 tablet_service.cc:1514] Processing DeleteTablet for tablet ad1a9f4b1c53434798ef9752e837c0dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:21 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39414
I20260808 04:45:21.452165 25528 tablet_service.cc:1514] Processing DeleteTablet for tablet ad1a9f4b1c53434798ef9752e837c0dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:21 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:51268
I20260808 04:45:21.452180 25753 tablet_service.cc:1514] Processing DeleteTablet for tablet ad1a9f4b1c53434798ef9752e837c0dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:21 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:38556
I20260808 04:45:21.452600 25104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 04:45:21.452690 25958 tablet_replica.cc:330] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: stopping tablet replica
I20260808 04:45:21.452737 25957 tablet_replica.cc:330] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: stopping tablet replica
I20260808 04:45:21.452994 25958 raft_consensus.cc:2229] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 04:45:21.453016 25104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:21.453001 25957 raft_consensus.cc:2229] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:21.453284 25958 raft_consensus.cc:2258] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:21.453385 25957 raft_consensus.cc:2258] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:21.454767 25958 ts_tablet_manager.cc:1905] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:21.454823 25957 ts_tablet_manager.cc:1905] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:21.457777 25958 ts_tablet_manager.cc:1918] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:21.457870 25958 log.cc:1196] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/ad1a9f4b1c53434798ef9752e837c0dd
I20260808 04:45:21.457947 25957 ts_tablet_manager.cc:1918] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:21.458039 25957 log.cc:1196] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/ad1a9f4b1c53434798ef9752e837c0dd
I20260808 04:45:21.458184 25958 ts_tablet_manager.cc:1939] T ad1a9f4b1c53434798ef9752e837c0dd P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting consensus metadata
I20260808 04:45:21.458423 25957 ts_tablet_manager.cc:1939] T ad1a9f4b1c53434798ef9752e837c0dd P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting consensus metadata
I20260808 04:45:21.458948 25753 tablet_service.cc:1467] Processing CreateTablet for tablet 4d2c654fd8b549e199b7e768df271d68 (DEFAULT_TABLE table=test-table [id=e163d49d983f4a92bc95099892a12557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:21.458959 25319 tablet_service.cc:1467] Processing CreateTablet for tablet 4d2c654fd8b549e199b7e768df271d68 (DEFAULT_TABLE table=test-table [id=e163d49d983f4a92bc95099892a12557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:21.459199 25753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d2c654fd8b549e199b7e768df271d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.459203 25319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d2c654fd8b549e199b7e768df271d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.459501 25081 catalog_manager.cc:4841] TS bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129:35075): tablet ad1a9f4b1c53434798ef9752e837c0dd (table test-table [id=16c9798128614d90b14862bd92af25df]) successfully deleted
I20260808 04:45:21.459785 25089 catalog_manager.cc:4841] TS 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251): tablet ad1a9f4b1c53434798ef9752e837c0dd (table test-table [id=16c9798128614d90b14862bd92af25df]) successfully deleted
I20260808 04:45:21.460134 25528 tablet_service.cc:1467] Processing CreateTablet for tablet 4d2c654fd8b549e199b7e768df271d68 (DEFAULT_TABLE table=test-table [id=e163d49d983f4a92bc95099892a12557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:21.460417 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d2c654fd8b549e199b7e768df271d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:21.461011 25890 tablet_bootstrap.cc:492] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: Bootstrap starting.
I20260808 04:45:21.461450 25888 tablet_bootstrap.cc:492] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Bootstrap starting.
I20260808 04:45:21.461906 25890 tablet_bootstrap.cc:654] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.462294 25888 tablet_bootstrap.cc:654] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.462976 25889 tablet_bootstrap.cc:492] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: Bootstrap starting.
I20260808 04:45:21.463397 25961 tablet_replica.cc:330] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: stopping tablet replica
I20260808 04:45:21.463548 25888 tablet_bootstrap.cc:492] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: No bootstrap required, opened a new log
I20260808 04:45:21.463639 25888 ts_tablet_manager.cc:1397] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:21.463843 25961 raft_consensus.cc:2229] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:21.463912 25889 tablet_bootstrap.cc:654] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:21.464164 25888 raft_consensus.cc:348] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.464287 25888 raft_consensus.cc:374] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.464324 25888 raft_consensus.cc:729] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.464455 25888 consensus_queue.cc:260] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.464836 25888 ts_tablet_manager.cc:1428] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:21.465767 25890 tablet_bootstrap.cc:492] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: No bootstrap required, opened a new log
I20260808 04:45:21.465849 25890 ts_tablet_manager.cc:1397] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260808 04:45:21.466063 25889 tablet_bootstrap.cc:492] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: No bootstrap required, opened a new log
I20260808 04:45:21.466135 25889 ts_tablet_manager.cc:1397] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:21.466634 25889 raft_consensus.cc:348] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.466737 25889 raft_consensus.cc:374] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.466776 25889 raft_consensus.cc:729] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 382e1f31e34441b28d28e343da52bbc8, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.466275 25890 raft_consensus.cc:348] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.466909 25889 consensus_queue.cc:260] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.466948 25890 raft_consensus.cc:374] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:21.466991 25890 raft_consensus.cc:729] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ea4a9a76f4ea9952cdcba9588f94e, State: Initialized, Role: FOLLOWER
I20260808 04:45:21.467108 25890 consensus_queue.cc:260] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.467180 25889 ts_tablet_manager.cc:1428] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:45:21.467352 25890 ts_tablet_manager.cc:1428] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:45:21.464865 25961 raft_consensus.cc:2258] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:21.469035 25961 ts_tablet_manager.cc:1905] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:21.471526 25961 ts_tablet_manager.cc:1918] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:21.471629 25961 log.cc:1196] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/ad1a9f4b1c53434798ef9752e837c0dd
I20260808 04:45:21.471998 25961 ts_tablet_manager.cc:1939] T ad1a9f4b1c53434798ef9752e837c0dd P 382e1f31e34441b28d28e343da52bbc8: Deleting consensus metadata
I20260808 04:45:21.473124 25089 catalog_manager.cc:4841] TS 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857): tablet ad1a9f4b1c53434798ef9752e837c0dd (table test-table [id=16c9798128614d90b14862bd92af25df]) successfully deleted
I20260808 04:45:21.660737 25954 raft_consensus.cc:482] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:21.660874 25954 raft_consensus.cc:504] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.661496 25954 leader_election.cc:288] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:21.662133 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d2c654fd8b549e199b7e768df271d68" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8" is_pre_election: true
I20260808 04:45:21.662184 25756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d2c654fd8b549e199b7e768df271d68" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" is_pre_election: true
I20260808 04:45:21.662343 25756 raft_consensus.cc:2454] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:21.662338 25557 raft_consensus.cc:2454] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:21.662650 25250 leader_election.cc:302] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 382e1f31e34441b28d28e343da52bbc8, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:21.662894 25954 raft_consensus.cc:2781] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:21.662967 25954 raft_consensus.cc:482] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:21.662998 25954 raft_consensus.cc:3037] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.664027 25954 raft_consensus.cc:504] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.664368 25954 leader_election.cc:288] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Requested vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:21.664801 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d2c654fd8b549e199b7e768df271d68" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8"
I20260808 04:45:21.664857 25756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d2c654fd8b549e199b7e768df271d68" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e"
I20260808 04:45:21.664916 25557 raft_consensus.cc:3037] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.664983 25756 raft_consensus.cc:3037] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:21.666081 25756 raft_consensus.cc:2454] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:21.666081 25557 raft_consensus.cc:2454] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:21.666395 25250 leader_election.cc:302] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959ea4a9a76f4ea9952cdcba9588f94e, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:21.666580 25954 raft_consensus.cc:2781] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:21.666673 25954 raft_consensus.cc:686] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 LEADER]: Becoming Leader. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Running, Role: LEADER
I20260808 04:45:21.666836 25954 consensus_queue.cc:237] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:21.669127 25104 catalog_manager.cc:5495] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:21.718356 25756 raft_consensus.cc:1261] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:21.718379 25557 raft_consensus.cc:1261] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:21.718809 25896 consensus_queue.cc:1035] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:21.718993 25942 consensus_queue.cc:1035] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 04:45:21.731598 25934 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 04:45:23.292639 25104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:23.292830 25104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:23.295826 25104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: Sending DeleteTablet for 3 replicas of tablet 4d2c654fd8b549e199b7e768df271d68
I20260808 04:45:23.296350 25321 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d2c654fd8b549e199b7e768df271d68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:23 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39414
I20260808 04:45:23.296607 25753 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d2c654fd8b549e199b7e768df271d68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:23 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:38556
I20260808 04:45:23.296613 25528 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d2c654fd8b549e199b7e768df271d68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:23 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:51268
I20260808 04:45:23.297015 26001 tablet_replica.cc:330] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: stopping tablet replica
I20260808 04:45:23.297178 26001 raft_consensus.cc:2229] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:23.297489 26001 raft_consensus.cc:2258] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:23.297513 26000 tablet_replica.cc:330] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: stopping tablet replica
I20260808 04:45:23.297814 25999 tablet_replica.cc:330] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: stopping tablet replica
I20260808 04:45:23.297986 25999 raft_consensus.cc:2229] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:23.298157 26000 raft_consensus.cc:2229] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:23.298220 25999 raft_consensus.cc:2258] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:23.298415 26000 raft_consensus.cc:2258] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:23.298815 26001 ts_tablet_manager.cc:1905] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:23.298985 25999 ts_tablet_manager.cc:1905] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:23.299611 26000 ts_tablet_manager.cc:1905] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:23.302074 26001 ts_tablet_manager.cc:1918] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:45:23.302175 26001 log.cc:1196] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/4d2c654fd8b549e199b7e768df271d68
I20260808 04:45:23.302443 26001 ts_tablet_manager.cc:1939] T 4d2c654fd8b549e199b7e768df271d68 P 382e1f31e34441b28d28e343da52bbc8: Deleting consensus metadata
I20260808 04:45:23.302474 25999 ts_tablet_manager.cc:1918] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:45:23.302551 25999 log.cc:1196] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/4d2c654fd8b549e199b7e768df271d68
I20260808 04:45:23.302915 25999 ts_tablet_manager.cc:1939] T 4d2c654fd8b549e199b7e768df271d68 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting consensus metadata
I20260808 04:45:23.303310 25089 catalog_manager.cc:4841] TS 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857): tablet 4d2c654fd8b549e199b7e768df271d68 (table test-table [id=e163d49d983f4a92bc95099892a12557]) successfully deleted
I20260808 04:45:23.303926 25081 catalog_manager.cc:4841] TS bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129:35075): tablet 4d2c654fd8b549e199b7e768df271d68 (table test-table [id=e163d49d983f4a92bc95099892a12557]) successfully deleted
I20260808 04:45:23.305418 26000 ts_tablet_manager.cc:1918] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:45:23.305516 26000 log.cc:1196] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/4d2c654fd8b549e199b7e768df271d68
I20260808 04:45:23.305801 26000 ts_tablet_manager.cc:1939] T 4d2c654fd8b549e199b7e768df271d68 P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting consensus metadata
I20260808 04:45:23.306478 25089 catalog_manager.cc:4841] TS 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251): tablet 4d2c654fd8b549e199b7e768df271d68 (table test-table [id=e163d49d983f4a92bc95099892a12557]) successfully deleted
Aug 08 04:45:23 dist-test-slave-04bb krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164323, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:45:23 dist-test-slave-04bb krb5kdc[25035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164323, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:23.326069 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:23.317277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56100 (local address 127.22.2.190:36981)
0808 04:45:23.317562 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:23.317568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:23.317583 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:23.317696 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:23.317698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:23.317754 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:23.317829 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:23.318572 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:23.319212 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:23.320049 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:23.320279 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:23.320398 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:23.322748 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:23.322751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:23.322753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:23.322776 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:23.324293 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:23.324811 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:23.324815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:23.324816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:23.324894 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:23.325181 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:23.325184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:23.325186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:23.325366 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:23.325508 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:23.325759 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 04:45:23.325885 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":135,"threads_started":1}
I20260808 04:45:23.328958 25104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:23.329407 25104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:23.334188 25321 tablet_service.cc:1467] Processing CreateTablet for tablet b6b7ba4074454deca787102faa9b36cb (DEFAULT_TABLE table=test-table [id=c592ebb7179d44d18fb1d4bcf5072318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:23.334208 25528 tablet_service.cc:1467] Processing CreateTablet for tablet b6b7ba4074454deca787102faa9b36cb (DEFAULT_TABLE table=test-table [id=c592ebb7179d44d18fb1d4bcf5072318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:23.334520 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b7ba4074454deca787102faa9b36cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:23.334535 25321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b7ba4074454deca787102faa9b36cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:23.336695 26015 tablet_bootstrap.cc:492] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Bootstrap starting.
I20260808 04:45:23.336961 25753 tablet_service.cc:1467] Processing CreateTablet for tablet b6b7ba4074454deca787102faa9b36cb (DEFAULT_TABLE table=test-table [id=c592ebb7179d44d18fb1d4bcf5072318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:23.337106 26014 tablet_bootstrap.cc:492] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: Bootstrap starting.
I20260808 04:45:23.337257 25753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b7ba4074454deca787102faa9b36cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:23.337693 26015 tablet_bootstrap.cc:654] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:23.338366 26014 tablet_bootstrap.cc:654] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:23.339709 26014 tablet_bootstrap.cc:492] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: No bootstrap required, opened a new log
I20260808 04:45:23.339815 26014 ts_tablet_manager.cc:1397] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:23.340315 26014 raft_consensus.cc:348] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.340448 26014 raft_consensus.cc:374] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:23.340481 26014 raft_consensus.cc:729] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 382e1f31e34441b28d28e343da52bbc8, State: Initialized, Role: FOLLOWER
I20260808 04:45:23.340647 26014 consensus_queue.cc:260] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.341105 26014 ts_tablet_manager.cc:1428] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:45:23.341394 26019 tablet_bootstrap.cc:492] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: Bootstrap starting.
I20260808 04:45:23.341684 26015 tablet_bootstrap.cc:492] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: No bootstrap required, opened a new log
I20260808 04:45:23.341777 26015 ts_tablet_manager.cc:1397] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 04:45:23.342283 26019 tablet_bootstrap.cc:654] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:23.342203 26015 raft_consensus.cc:348] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.342329 26015 raft_consensus.cc:374] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:23.342361 26015 raft_consensus.cc:729] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Initialized, Role: FOLLOWER
I20260808 04:45:23.342484 26015 consensus_queue.cc:260] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.342785 26015 ts_tablet_manager.cc:1428] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:45:23.343717 26019 tablet_bootstrap.cc:492] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: No bootstrap required, opened a new log
I20260808 04:45:23.343832 26019 ts_tablet_manager.cc:1397] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:23.344223 26019 raft_consensus.cc:348] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.344341 26019 raft_consensus.cc:374] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:23.344378 26019 raft_consensus.cc:729] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ea4a9a76f4ea9952cdcba9588f94e, State: Initialized, Role: FOLLOWER
I20260808 04:45:23.344540 26019 consensus_queue.cc:260] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.344967 26019 ts_tablet_manager.cc:1428] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:23.520745 25979 raft_consensus.cc:482] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:23.520959 25979 raft_consensus.cc:504] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.521471 25979 leader_election.cc:288] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:23.522003 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b7ba4074454deca787102faa9b36cb" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8" is_pre_election: true
I20260808 04:45:23.522080 25786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b7ba4074454deca787102faa9b36cb" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" is_pre_election: true
I20260808 04:45:23.522183 25557 raft_consensus.cc:2454] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:23.522213 25786 raft_consensus.cc:2454] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:23.522567 25250 leader_election.cc:302] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959ea4a9a76f4ea9952cdcba9588f94e, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:23.522770 25979 raft_consensus.cc:2781] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:23.522845 25979 raft_consensus.cc:482] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:23.522889 25979 raft_consensus.cc:3037] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:23.523900 25979 raft_consensus.cc:504] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.524228 25979 leader_election.cc:288] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Requested vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:23.524657 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b7ba4074454deca787102faa9b36cb" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8"
I20260808 04:45:23.524719 25786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b7ba4074454deca787102faa9b36cb" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e"
I20260808 04:45:23.524806 25557 raft_consensus.cc:3037] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:23.524852 25786 raft_consensus.cc:3037] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:23.525905 25557 raft_consensus.cc:2454] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:23.526023 25786 raft_consensus.cc:2454] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:23.526253 25250 leader_election.cc:302] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 382e1f31e34441b28d28e343da52bbc8, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:23.526465 25979 raft_consensus.cc:2781] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:23.526602 25979 raft_consensus.cc:686] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 LEADER]: Becoming Leader. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Running, Role: LEADER
I20260808 04:45:23.526758 25979 consensus_queue.cc:237] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.528505 25101 catalog_manager.cc:5495] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:23.582448 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:23.579168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39436 (local address 127.22.2.129:35075)
0808 04:45:23.579460 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:23.579468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:23.579482 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:45:23.579527 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:23.579530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:23.579578 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:23.579663 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:23.579886 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:23.580560 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:23.581365 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:23.581585 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:23.581701 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:23.581993 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:23.582065 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:23.582146 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:23.582244 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 04:45:23.582296 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":130,"threads_started":1}
I20260808 04:45:23.589103 25786 raft_consensus.cc:1261] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:23.589169 25557 raft_consensus.cc:1261] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:23.589493 25979 consensus_queue.cc:1035] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:23.589641 25942 consensus_queue.cc:1035] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:23.619249 25101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:23.619465 25101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:23.621587 25101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: Sending DeleteTablet for 3 replicas of tablet b6b7ba4074454deca787102faa9b36cb
I20260808 04:45:23.622187 25319 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b7ba4074454deca787102faa9b36cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:23 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39414
I20260808 04:45:23.622191 25528 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b7ba4074454deca787102faa9b36cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:23 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:51268
I20260808 04:45:23.622332 25753 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b7ba4074454deca787102faa9b36cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:23 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:38556
I20260808 04:45:23.622392 25999 tablet_replica.cc:330] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: stopping tablet replica
I20260808 04:45:23.622409 26001 tablet_replica.cc:330] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: stopping tablet replica
I20260808 04:45:23.622495 26000 tablet_replica.cc:330] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: stopping tablet replica
I20260808 04:45:23.622510 25999 raft_consensus.cc:2229] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:23.622547 26001 raft_consensus.cc:2229] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:23.622601 26000 raft_consensus.cc:2229] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:23.622669 26001 raft_consensus.cc:2258] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:23.622706 26000 raft_consensus.cc:2258] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:23.622694 25999 raft_consensus.cc:2258] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:23.623349 25999 ts_tablet_manager.cc:1905] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:23.623466 26000 ts_tablet_manager.cc:1905] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:23.624033 25101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:23.624439 25101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:23.625896 26000 ts_tablet_manager.cc:1918] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:23.625991 26000 log.cc:1196] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/b6b7ba4074454deca787102faa9b36cb
I20260808 04:45:23.626245 26000 ts_tablet_manager.cc:1939] T b6b7ba4074454deca787102faa9b36cb P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting consensus metadata
I20260808 04:45:23.626251 25999 ts_tablet_manager.cc:1918] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:23.626341 25999 log.cc:1196] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/b6b7ba4074454deca787102faa9b36cb
I20260808 04:45:23.626680 25999 ts_tablet_manager.cc:1939] T b6b7ba4074454deca787102faa9b36cb P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Deleting consensus metadata
I20260808 04:45:23.626958 25089 catalog_manager.cc:4841] TS 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251): tablet b6b7ba4074454deca787102faa9b36cb (table test-table [id=c592ebb7179d44d18fb1d4bcf5072318]) successfully deleted
I20260808 04:45:23.627350 25081 catalog_manager.cc:4841] TS bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129:35075): tablet b6b7ba4074454deca787102faa9b36cb (table test-table [id=c592ebb7179d44d18fb1d4bcf5072318]) successfully deleted
I20260808 04:45:23.628285 26001 ts_tablet_manager.cc:1905] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:23.630793 26001 ts_tablet_manager.cc:1918] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:23.630883 26001 log.cc:1196] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/b6b7ba4074454deca787102faa9b36cb
I20260808 04:45:23.631171 26001 ts_tablet_manager.cc:1939] T b6b7ba4074454deca787102faa9b36cb P 382e1f31e34441b28d28e343da52bbc8: Deleting consensus metadata
I20260808 04:45:23.631352 25319 tablet_service.cc:1467] Processing CreateTablet for tablet abbf358bf9484fdd98f950dfc0f76942 (DEFAULT_TABLE table=test-table [id=841ff9002a914d15a2713d63df71ce94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:23.631593 25319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abbf358bf9484fdd98f950dfc0f76942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:23.631670 25753 tablet_service.cc:1467] Processing CreateTablet for tablet abbf358bf9484fdd98f950dfc0f76942 (DEFAULT_TABLE table=test-table [id=841ff9002a914d15a2713d63df71ce94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:23.631830 25089 catalog_manager.cc:4841] TS 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857): tablet b6b7ba4074454deca787102faa9b36cb (table test-table [id=c592ebb7179d44d18fb1d4bcf5072318]) successfully deleted
I20260808 04:45:23.631953 25753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abbf358bf9484fdd98f950dfc0f76942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:23.633606 26015 tablet_bootstrap.cc:492] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Bootstrap starting.
I20260808 04:45:23.633810 26019 tablet_bootstrap.cc:492] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: Bootstrap starting.
I20260808 04:45:23.633816 25528 tablet_service.cc:1467] Processing CreateTablet for tablet abbf358bf9484fdd98f950dfc0f76942 (DEFAULT_TABLE table=test-table [id=841ff9002a914d15a2713d63df71ce94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:23.634038 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abbf358bf9484fdd98f950dfc0f76942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:23.634429 26015 tablet_bootstrap.cc:654] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:23.634704 26019 tablet_bootstrap.cc:654] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:23.635895 26019 tablet_bootstrap.cc:492] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: No bootstrap required, opened a new log
I20260808 04:45:23.635954 26019 ts_tablet_manager.cc:1397] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:23.636106 26015 tablet_bootstrap.cc:492] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: No bootstrap required, opened a new log
I20260808 04:45:23.636191 26015 ts_tablet_manager.cc:1397] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 04:45:23.636199 26014 tablet_bootstrap.cc:492] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: Bootstrap starting.
I20260808 04:45:23.636376 26019 raft_consensus.cc:348] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.636487 26019 raft_consensus.cc:374] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:23.636543 26019 raft_consensus.cc:729] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ea4a9a76f4ea9952cdcba9588f94e, State: Initialized, Role: FOLLOWER
I20260808 04:45:23.636626 26015 raft_consensus.cc:348] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.636685 26019 consensus_queue.cc:260] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.636762 26015 raft_consensus.cc:374] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:23.636811 26015 raft_consensus.cc:729] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Initialized, Role: FOLLOWER
I20260808 04:45:23.636937 26015 consensus_queue.cc:260] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.636976 26019 ts_tablet_manager.cc:1428] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:23.637173 26014 tablet_bootstrap.cc:654] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:23.637192 26015 ts_tablet_manager.cc:1428] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:23.638772 26014 tablet_bootstrap.cc:492] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: No bootstrap required, opened a new log
I20260808 04:45:23.638865 26014 ts_tablet_manager.cc:1397] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 04:45:23.639310 26014 raft_consensus.cc:348] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.639428 26014 raft_consensus.cc:374] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:23.639467 26014 raft_consensus.cc:729] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 382e1f31e34441b28d28e343da52bbc8, State: Initialized, Role: FOLLOWER
I20260808 04:45:23.639592 26014 consensus_queue.cc:260] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.639925 26014 ts_tablet_manager.cc:1428] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 04:45:23.732545 25973 raft_consensus.cc:482] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:23.732709 25973 raft_consensus.cc:504] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.733167 25973 leader_election.cc:288] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:23.733664 25756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abbf358bf9484fdd98f950dfc0f76942" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" is_pre_election: true
I20260808 04:45:23.733654 25556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abbf358bf9484fdd98f950dfc0f76942" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8" is_pre_election: true
I20260808 04:45:23.733803 25756 raft_consensus.cc:2454] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:23.733845 25556 raft_consensus.cc:2454] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 0.
I20260808 04:45:23.734172 25250 leader_election.cc:302] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959ea4a9a76f4ea9952cdcba9588f94e, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:23.734388 25973 raft_consensus.cc:2781] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:23.734472 25973 raft_consensus.cc:482] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:23.734534 25973 raft_consensus.cc:3037] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:23.735724 25973 raft_consensus.cc:504] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.736089 25973 leader_election.cc:288] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Requested vote from peers 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857), 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251)
I20260808 04:45:23.736469 25556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abbf358bf9484fdd98f950dfc0f76942" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "382e1f31e34441b28d28e343da52bbc8"
I20260808 04:45:23.736603 25756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abbf358bf9484fdd98f950dfc0f76942" candidate_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ea4a9a76f4ea9952cdcba9588f94e"
I20260808 04:45:23.736685 25556 raft_consensus.cc:3037] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:23.736723 25756 raft_consensus.cc:3037] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:23.737639 25756 raft_consensus.cc:2454] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:23.737743 25556 raft_consensus.cc:2454] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcbb3f6f9dfe44f98ff3b71ef690bbd2 in term 1.
I20260808 04:45:23.737973 25250 leader_election.cc:302] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959ea4a9a76f4ea9952cdcba9588f94e, bcbb3f6f9dfe44f98ff3b71ef690bbd2; no voters:
I20260808 04:45:23.738183 25973 raft_consensus.cc:2781] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:23.738287 25973 raft_consensus.cc:686] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [term 1 LEADER]: Becoming Leader. State: Replica: bcbb3f6f9dfe44f98ff3b71ef690bbd2, State: Running, Role: LEADER
I20260808 04:45:23.738445 25973 consensus_queue.cc:237] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } }
I20260808 04:45:23.740264 25108 catalog_manager.cc:5495] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbb3f6f9dfe44f98ff3b71ef690bbd2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:23.768054 25556 raft_consensus.cc:1261] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:23.768333 25756 raft_consensus.cc:1261] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Refusing update from remote peer bcbb3f6f9dfe44f98ff3b71ef690bbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:23.768512 25942 consensus_queue.cc:1035] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "382e1f31e34441b28d28e343da52bbc8" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:23.768729 25973 consensus_queue.cc:1035] T abbf358bf9484fdd98f950dfc0f76942 P bcbb3f6f9dfe44f98ff3b71ef690bbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ea4a9a76f4ea9952cdcba9588f94e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 04:45:23.782531 25971 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 04:45:25.340751 25101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:25.340893 25101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:25.344161 25101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61bb630d7f724d0295d60cdbab38e675: Sending DeleteTablet for 3 replicas of tablet abbf358bf9484fdd98f950dfc0f76942
I20260808 04:45:25.344748 25319 tablet_service.cc:1514] Processing DeleteTablet for tablet abbf358bf9484fdd98f950dfc0f76942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:25 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39414
I20260808 04:45:25.345044 25528 tablet_service.cc:1514] Processing DeleteTablet for tablet abbf358bf9484fdd98f950dfc0f76942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:25 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:51268
I20260808 04:45:25.345366 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 25217
I20260808 04:45:25.345372 25753 tablet_service.cc:1514] Processing DeleteTablet for tablet abbf358bf9484fdd98f950dfc0f76942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:25 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:38556
I20260808 04:45:25.352384 26078 tablet_replica.cc:330] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: stopping tablet replica
I20260808 04:45:25.352507 26078 raft_consensus.cc:2229] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:25.352691 26078 raft_consensus.cc:2258] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:25.352778 26079 tablet_replica.cc:330] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: stopping tablet replica
I20260808 04:45:25.352926 26079 raft_consensus.cc:2229] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:25.353053 26079 raft_consensus.cc:2258] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:25.353528 26078 ts_tablet_manager.cc:1905] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:25.354671 26079 ts_tablet_manager.cc:1905] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:25.356920 26079 ts_tablet_manager.cc:1918] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:45:25.357015 26079 log.cc:1196] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/abbf358bf9484fdd98f950dfc0f76942
I20260808 04:45:25.357086 26078 ts_tablet_manager.cc:1918] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:45:25.357180 26078 log.cc:1196] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/abbf358bf9484fdd98f950dfc0f76942
I20260808 04:45:25.357393 26079 ts_tablet_manager.cc:1939] T abbf358bf9484fdd98f950dfc0f76942 P 959ea4a9a76f4ea9952cdcba9588f94e: Deleting consensus metadata
I20260808 04:45:25.357501 26078 ts_tablet_manager.cc:1939] T abbf358bf9484fdd98f950dfc0f76942 P 382e1f31e34441b28d28e343da52bbc8: Deleting consensus metadata
I20260808 04:45:25.358139 25089 catalog_manager.cc:4841] TS 959ea4a9a76f4ea9952cdcba9588f94e (127.22.2.131:42251): tablet abbf358bf9484fdd98f950dfc0f76942 (table test-table [id=841ff9002a914d15a2713d63df71ce94]) successfully deleted
I20260808 04:45:25.358362 25089 catalog_manager.cc:4841] TS 382e1f31e34441b28d28e343da52bbc8 (127.22.2.130:33857): tablet abbf358bf9484fdd98f950dfc0f76942 (table test-table [id=841ff9002a914d15a2713d63df71ce94]) successfully deleted
W20260808 04:45:25.362870 25081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:35075 (error 108)
I20260808 04:45:25.363667 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 25426
I20260808 04:45:25.366034 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:25.363684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35075 (local address 127.0.0.1:38234)
0808 04:45:25.365788 (+ 2104us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:25.365895 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2002,"thread_start_us":120,"threads_started":1}
W20260808 04:45:25.366155 25081 catalog_manager.cc:4584] TS bcbb3f6f9dfe44f98ff3b71ef690bbd2 (127.22.2.129:35075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet abbf358bf9484fdd98f950dfc0f76942: Network error: Client connection negotiation failed: client connection to 127.22.2.129:35075: connect: Connection refused (error 111)
I20260808 04:45:25.379611 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 25647
I20260808 04:45:25.387144 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 25054
2026-08-08T04:45:25Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6683 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:25 dist-test-slave-04bb krb5kdc[26087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:25 dist-test-slave-04bb krb5kdc[26087](info): set up 2 sockets
Aug 08 04:45:25 dist-test-slave-04bb krb5kdc[26087](info): commencing operation
krb5kdc: starting...
W20260808 04:45:26.636837 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:26 dist-test-slave-04bb krb5kdc[26087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:26.763312 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36159
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:36159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:26.877722 26107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:26.877969 26107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:26.878006 26107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:26.881855 26107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:26.881904 26107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:26.881923 26107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:26.881939 26107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:26.881955 26107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:26.887094 26107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:36159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:26.888085 26107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:26.889060 26107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:26.898294 26126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:26.898301 26123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:26.898380 26107 server_base.cc:1034] running on GCE node
W20260808 04:45:26.898290 26121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:26.899250 26107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:26.900452 26107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:26.901638 26107 hybrid_clock.cc:648] HybridClock initialized: now 1786164326901609 us; error 43 us; skew 500 ppm
Aug 08 04:45:26 dist-test-slave-04bb krb5kdc[26087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164326, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:26.908429 26107 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:26.909435 26107 webserver.cc:460] Webserver started at http://127.22.2.190:35433/ using document root <none> and password file <none>
I20260808 04:45:26.910037 26107 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:26.910132 26107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:26.910388 26107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:26.912442 26107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "697541baadb440f994bc750e8cfc2a66"
format_stamp: "Formatted at 2026-08-08 04:45:26 on dist-test-slave-04bb"
server_key: "aefa3656672d049f7897894224a29b67"
server_key_iv: "77fcf5d1de23dbbd8ddcf4ddbecddf20"
server_key_version: "encryptionkey@0"
I20260808 04:45:26.913081 26107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "697541baadb440f994bc750e8cfc2a66"
format_stamp: "Formatted at 2026-08-08 04:45:26 on dist-test-slave-04bb"
server_key: "aefa3656672d049f7897894224a29b67"
server_key_iv: "77fcf5d1de23dbbd8ddcf4ddbecddf20"
server_key_version: "encryptionkey@0"
I20260808 04:45:26.916865 26107 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:45:26.919373 26135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:26.920600 26107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:26.920748 26107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "697541baadb440f994bc750e8cfc2a66"
format_stamp: "Formatted at 2026-08-08 04:45:26 on dist-test-slave-04bb"
server_key: "aefa3656672d049f7897894224a29b67"
server_key_iv: "77fcf5d1de23dbbd8ddcf4ddbecddf20"
server_key_version: "encryptionkey@0"
I20260808 04:45:26.920859 26107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:26.945276 26107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:26.948697 26107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:26.948907 26107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:26.956462 26107 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:36159
I20260808 04:45:26.956485 26206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:36159 every 8 connection(s)
I20260808 04:45:26.957436 26107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:26.959970 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 26107
I20260808 04:45:26.960067 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:26.960354 22538 external_mini_cluster.cc:1428] Setting key 84d01c7c4d072eb552bda3680e88b14d
I20260808 04:45:26.960574 26207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:26.968431 26207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66: Bootstrap starting.
Aug 08 04:45:26 dist-test-slave-04bb krb5kdc[26087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:26.971293 26207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:26.972218 26207 log.cc:824] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:26.974273 26207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66: No bootstrap required, opened a new log
I20260808 04:45:26.975673 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:26.961869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.22.2.190:36159)
0808 04:45:26.962315 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:26.962335 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:26.962381 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:45:26.963165 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:26.963189 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:26.963626 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:26.964032 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:26.964755 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:26.966376 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:26.967226 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:26.968007 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:26.968185 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:26.970992 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:26.970997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:26.971010 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:26.971036 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:26.972863 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:26.973331 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:26.973336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:26.973341 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:26.973449 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:26.973759 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:26.973763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:26.973764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:26.974163 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:26.974366 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:26.974694 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 04:45:26.974965 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":123,"threads_started":1}
I20260808 04:45:26.978615 26207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697541baadb440f994bc750e8cfc2a66" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36159 } }
I20260808 04:45:26.978866 26207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:26.978966 26207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697541baadb440f994bc750e8cfc2a66, State: Initialized, Role: FOLLOWER
I20260808 04:45:26.979468 26207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697541baadb440f994bc750e8cfc2a66" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36159 } }
I20260808 04:45:26.979630 26207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:26.979730 26207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:26.979839 26207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:26.980906 26207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697541baadb440f994bc750e8cfc2a66" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36159 } }
I20260808 04:45:26.981307 26207 leader_election.cc:302] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 697541baadb440f994bc750e8cfc2a66; no voters:
I20260808 04:45:26.981681 26207 leader_election.cc:288] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:26.981798 26218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:26.982036 26218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [term 1 LEADER]: Becoming Leader. State: Replica: 697541baadb440f994bc750e8cfc2a66, State: Running, Role: LEADER
I20260808 04:45:26.982425 26218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697541baadb440f994bc750e8cfc2a66" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36159 } }
I20260808 04:45:26.982969 26207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:26.984133 26220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "697541baadb440f994bc750e8cfc2a66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697541baadb440f994bc750e8cfc2a66" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36159 } } }
I20260808 04:45:26.984277 26220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:26.984591 26223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 697541baadb440f994bc750e8cfc2a66. Latest consensus state: current_term: 1 leader_uuid: "697541baadb440f994bc750e8cfc2a66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697541baadb440f994bc750e8cfc2a66" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36159 } } }
I20260808 04:45:26.984673 26223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:26.985687 26233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:26.988445 26233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:26.993983 26233 catalog_manager.cc:1347] Generated new cluster ID: e8a204dc81d140baa0365121f867e048
I20260808 04:45:26.994066 26233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:27.002650 26233 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:27.003799 26233 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:27.010756 26233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 697541baadb440f994bc750e8cfc2a66: Generated new TSK 0
I20260808 04:45:27.011479 26233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:27.048425 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36159
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:27.165620 26249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:27.165844 26249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:27.165884 26249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:27.169818 26249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:27.169873 26249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:27.169929 26249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:27.175073 26249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:27.176051 26249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:27.177084 26249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:27.186741 26256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:27.186741 26257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:27.186831 26260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:27.187038 26249 server_base.cc:1034] running on GCE node
I20260808 04:45:27.187450 26249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:27.188100 26249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:27.189287 26249 hybrid_clock.cc:648] HybridClock initialized: now 1786164327189273 us; error 25 us; skew 500 ppm
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:27.192091 26249 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:27.193065 26249 webserver.cc:460] Webserver started at http://127.22.2.129:35143/ using document root <none> and password file <none>
I20260808 04:45:27.193600 26249 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:27.193658 26249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:27.193823 26249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:27.195784 26249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "cafdcf74340f46a9b194fd3305434c1d"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "5a77ae4f8049cfb36b94269691d6ad46"
server_key_iv: "583dc88dde6b7bb51a6ffc5b170259f2"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.196300 26249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "cafdcf74340f46a9b194fd3305434c1d"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "5a77ae4f8049cfb36b94269691d6ad46"
server_key_iv: "583dc88dde6b7bb51a6ffc5b170259f2"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.200047 26249 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:27.202708 26269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.203922 26249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:27.204031 26249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "cafdcf74340f46a9b194fd3305434c1d"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "5a77ae4f8049cfb36b94269691d6ad46"
server_key_iv: "583dc88dde6b7bb51a6ffc5b170259f2"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.204166 26249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:27.233047 26249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:27.236588 26249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:27.236800 26249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:27.237416 26249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:27.238255 26249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:27.238301 26249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.238391 26249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:27.238423 26249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.247597 26249 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:39053
I20260808 04:45:27.247627 26442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:39053 every 8 connection(s)
I20260808 04:45:27.248513 26249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:27.255846 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 26249
I20260808 04:45:27.255970 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:27.256249 22538 external_mini_cluster.cc:1428] Setting key 705d8465aa63e59941be0cbcbbfc876c
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:27.261826 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:27.250301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:35393 (local address 127.22.2.190:36159)
0808 04:45:27.250437 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:27.250440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:27.251361 (+ 921us) server_negotiation.cc:407] Connection header received
0808 04:45:27.252155 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:27.252158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:27.252207 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:27.252281 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:27.253402 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.253942 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.255015 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.255234 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.255308 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:27.258175 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:27.258179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:27.258182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:27.258207 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:27.259675 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:27.260199 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:27.260203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:27.260205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:27.260296 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:27.260701 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:27.260705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:27.260707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:27.260904 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:27.261022 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:27.261520 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 04:45:27.261645 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 04:45:27.262570 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:27.250584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36159 (local address 127.22.2.129:35393)
0808 04:45:27.251208 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:27.251249 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:27.251977 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:27.252446 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:27.252452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:27.252814 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:27.253264 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:27.253277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.254050 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.254053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:27.254907 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:27.254915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.255357 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.255360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:27.255489 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:27.256136 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:27.256152 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:27.258009 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:27.259803 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:27.259808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:27.259822 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:27.260087 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:27.260394 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:27.260398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:27.260400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:27.260594 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:27.261020 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:27.261027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:27.261261 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 04:45:27.261460 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 04:45:27.261762 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":111,"threads_started":1}
I20260808 04:45:27.263798 26443 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36159
I20260808 04:45:27.264070 26443 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:27.264724 26443 heartbeater.cc:511] Master 127.22.2.190:36159 requested a full tablet report, sending...
I20260808 04:45:27.266309 26163 ts_manager.cc:194] Registered new tserver with Master: cafdcf74340f46a9b194fd3305434c1d (127.22.2.129:39053)
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:45:27.267495 26163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:35393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:27.285540 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36159
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:27.400463 26448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:27.400789 26448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:27.400833 26448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:27.404808 26448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:27.404855 26448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:27.404944 26448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:27.410252 26448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:27.411270 26448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:27.412297 26448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:27.421885 26453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:27.421916 26456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:27.422013 26448 server_base.cc:1034] running on GCE node
W20260808 04:45:27.421921 26454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:27.422796 26448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:27.423468 26448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:27.424626 26448 hybrid_clock.cc:648] HybridClock initialized: now 1786164327424583 us; error 54 us; skew 500 ppm
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:27.427474 26448 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:27.428395 26448 webserver.cc:460] Webserver started at http://127.22.2.130:39475/ using document root <none> and password file <none>
I20260808 04:45:27.429008 26448 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:27.429100 26448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:27.429322 26448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:27.431310 26448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "48025557cae64d1e9463b923879ed374"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "811d71460a1488c31ec9bb58ecd4630a"
server_key_iv: "79a07d342c38e42b531274e0af0c552f"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.431862 26448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "48025557cae64d1e9463b923879ed374"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "811d71460a1488c31ec9bb58ecd4630a"
server_key_iv: "79a07d342c38e42b531274e0af0c552f"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.435542 26448 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:45:27.437909 26470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.439023 26448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:27.439169 26448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "48025557cae64d1e9463b923879ed374"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "811d71460a1488c31ec9bb58ecd4630a"
server_key_iv: "79a07d342c38e42b531274e0af0c552f"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.439306 26448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:27.454949 26448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:27.458142 26448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:27.458343 26448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:27.458900 26448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:27.459705 26448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:27.459777 26448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.459846 26448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:27.459900 26448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.469043 26448 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:38419
I20260808 04:45:27.469069 26634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:38419 every 8 connection(s)
I20260808 04:45:27.469919 26448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:27.471736 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 26448
I20260808 04:45:27.471854 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:27.472167 22538 external_mini_cluster.cc:1428] Setting key ab375b6c203ea2e934e39172c6fe4920
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:45:27.485538 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:27.471858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:40113 (local address 127.22.2.190:36159)
0808 04:45:27.472703 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:27.472708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:27.473015 (+ 307us) server_negotiation.cc:407] Connection header received
0808 04:45:27.474534 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:27.474537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:27.474590 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:27.474696 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:27.476239 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.477122 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.478271 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.478497 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.478612 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:27.481579 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:27.481582 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:27.481584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:27.481608 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:27.483067 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:27.483733 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:27.483736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:27.483738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:27.483835 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:27.484355 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:27.484359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:27.484361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:27.484620 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:27.484753 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:27.485237 (+ 484us) server_negotiation.cc:299] Negotiation successful
0808 04:45:27.485413 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:45:27.486613 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:27.472305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36159 (local address 127.22.2.130:40113)
0808 04:45:27.472881 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:27.472923 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:27.474313 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:27.474908 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:27.474919 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:27.475375 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:27.476056 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:27.476080 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.477260 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.477263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:27.478134 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:27.478143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.478651 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.478655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:27.478905 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:27.479543 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:27.479561 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:27.481400 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:27.483222 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:27.483233 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:27.483249 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:27.483624 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:27.483938 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:27.483940 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:27.483942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:27.484243 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:27.484729 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:27.484735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:27.484963 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 04:45:27.485168 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 04:45:27.485537 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":170,"threads_started":1}
I20260808 04:45:27.487833 26636 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36159
I20260808 04:45:27.488094 26636 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:27.488648 26636 heartbeater.cc:511] Master 127.22.2.190:36159 requested a full tablet report, sending...
I20260808 04:45:27.489778 26163 ts_manager.cc:194] Registered new tserver with Master: 48025557cae64d1e9463b923879ed374 (127.22.2.130:38419)
I20260808 04:45:27.490386 26163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:40113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:27.503839 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36159
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:27.618290 26643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:27.618577 26643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:27.618659 26643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:27.622936 26643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:27.623047 26643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:27.623164 26643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:27.629143 26643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:27.630249 26643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:27.631265 26643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:27.641394 26660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:27.641394 26652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:27.641368 26654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:27.641670 26643 server_base.cc:1034] running on GCE node
I20260808 04:45:27.642155 26643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:27.642848 26643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:27.644019 26643 hybrid_clock.cc:648] HybridClock initialized: now 1786164327644005 us; error 28 us; skew 500 ppm
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:27.646972 26643 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:27.647832 26643 webserver.cc:460] Webserver started at http://127.22.2.131:46601/ using document root <none> and password file <none>
I20260808 04:45:27.648401 26643 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:27.648494 26643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:27.648761 26643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:27.651170 26643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "564ea57dce2445229668e2a29b9606e2"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "2ad455868af34fe01d0d2f2150cd84a7"
server_key_iv: "f2b1403b6c2b4c7e676aca1fccc4d900"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.651835 26643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "564ea57dce2445229668e2a29b9606e2"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "2ad455868af34fe01d0d2f2150cd84a7"
server_key_iv: "f2b1403b6c2b4c7e676aca1fccc4d900"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.656013 26643 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 04:45:27.658541 26671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.659757 26643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:27.659895 26643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "564ea57dce2445229668e2a29b9606e2"
format_stamp: "Formatted at 2026-08-08 04:45:27 on dist-test-slave-04bb"
server_key: "2ad455868af34fe01d0d2f2150cd84a7"
server_key_iv: "f2b1403b6c2b4c7e676aca1fccc4d900"
server_key_version: "encryptionkey@0"
I20260808 04:45:27.660001 26643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:27.684595 26643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:27.688692 26643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:27.688906 26643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:27.689791 26643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:27.690727 26643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:27.690778 26643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.690843 26643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:27.690877 26643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:27.699596 26643 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:33779
I20260808 04:45:27.699661 26834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:33779 every 8 connection(s)
I20260808 04:45:27.700601 26643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:27.700809 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 26643
I20260808 04:45:27.700919 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:27.701296 22538 external_mini_cluster.cc:1428] Setting key 00fe7faca0d965ca3727050b7ae7ae8d
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:27.714511 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:27.702691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:49533 (local address 127.22.2.190:36159)
0808 04:45:27.702839 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:27.702842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:27.703926 (+ 1084us) server_negotiation.cc:407] Connection header received
0808 04:45:27.705370 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:27.705373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:27.705419 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:27.705480 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:27.706562 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.707076 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.708274 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.708493 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.708651 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:27.710968 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:27.710977 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:27.710979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:27.711003 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:27.712334 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:27.712924 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:27.712928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:27.712930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:27.713011 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:27.713356 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:27.713359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:27.713361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:27.713540 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:27.713652 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:27.714224 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 04:45:27.714342 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:27.715247 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:27.703162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36159 (local address 127.22.2.131:49533)
0808 04:45:27.703767 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:27.703831 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:27.705170 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:27.705608 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:27.705614 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:27.705968 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:27.706437 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:27.706450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.707175 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.707177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:27.708152 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:27.708163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:27.708677 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:27.708680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:27.708827 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:27.709349 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:27.709364 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:27.710810 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:27.712451 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:27.712456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:27.712471 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:27.712822 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:27.713112 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:27.713116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:27.713118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:27.713258 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:27.713653 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:27.713660 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:27.713957 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 04:45:27.714165 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 04:45:27.714464 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":147,"threads_started":1}
I20260808 04:45:27.716266 26835 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36159
I20260808 04:45:27.716504 26835 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:27.716993 26835 heartbeater.cc:511] Master 127.22.2.190:36159 requested a full tablet report, sending...
I20260808 04:45:27.717973 26163 ts_manager.cc:194] Registered new tserver with Master: 564ea57dce2445229668e2a29b9606e2 (127.22.2.131:33779)
I20260808 04:45:27.718528 26163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:49533
I20260808 04:45:27.726881 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 04:45:27.741221 26214 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:27.739891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.22.2.190:36159)
0808 04:45:27.740010 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:27.740014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:27.740176 (+ 162us) server_negotiation.cc:407] Connection header received
0808 04:45:27.740283 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:27.740286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:27.740327 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:27.740399 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:27.741113 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41716: BlockingRecv error: recv got EOF from 127.0.0.1:41716 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 04:45:27.742239 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 26249
I20260808 04:45:27.762327 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 26448
I20260808 04:45:27.770836 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 26643
I20260808 04:45:27.780687 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 26107
2026-08-08T04:45:27Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2392 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26851](info): set up 2 sockets
Aug 08 04:45:27 dist-test-slave-04bb krb5kdc[26851](info): commencing operation
krb5kdc: starting...
W20260808 04:45:29.015282 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:29 dist-test-slave-04bb krb5kdc[26851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:29.143484 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34029
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:34029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:29.256310 26871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:29.256564 26871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:29.256603 26871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:29.260480 26871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:29.260572 26871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:29.260614 26871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:29.260635 26871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:29.260651 26871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:29.265657 26871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:34029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:29.266602 26871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:29.267582 26871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:29.275946 26878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:29.276000 26881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:29.275983 26879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:29.276134 26871 server_base.cc:1034] running on GCE node
I20260808 04:45:29.276958 26871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:29.278004 26871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:29.279218 26871 hybrid_clock.cc:648] HybridClock initialized: now 1786164329279201 us; error 31 us; skew 500 ppm
Aug 08 04:45:29 dist-test-slave-04bb krb5kdc[26851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164329, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:29.282023 26871 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:29.282899 26871 webserver.cc:460] Webserver started at http://127.22.2.190:41011/ using document root <none> and password file <none>
I20260808 04:45:29.283463 26871 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:29.283552 26871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:29.283789 26871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:29.285821 26871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "e7b8b09904644befb0608c78b592a9f3"
format_stamp: "Formatted at 2026-08-08 04:45:29 on dist-test-slave-04bb"
server_key: "a2a4107147d69e2f115aa03e896aafe1"
server_key_iv: "f181c3657626508e9c89f2554d90480c"
server_key_version: "encryptionkey@0"
I20260808 04:45:29.286387 26871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "e7b8b09904644befb0608c78b592a9f3"
format_stamp: "Formatted at 2026-08-08 04:45:29 on dist-test-slave-04bb"
server_key: "a2a4107147d69e2f115aa03e896aafe1"
server_key_iv: "f181c3657626508e9c89f2554d90480c"
server_key_version: "encryptionkey@0"
I20260808 04:45:29.290104 26871 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:29.292470 26899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:29.293735 26871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:29.293877 26871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "e7b8b09904644befb0608c78b592a9f3"
format_stamp: "Formatted at 2026-08-08 04:45:29 on dist-test-slave-04bb"
server_key: "a2a4107147d69e2f115aa03e896aafe1"
server_key_iv: "f181c3657626508e9c89f2554d90480c"
server_key_version: "encryptionkey@0"
I20260808 04:45:29.294000 26871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:29.330255 26871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:29.333706 26871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:29.333914 26871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:29.341253 26871 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:34029
I20260808 04:45:29.341604 26978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:34029 every 8 connection(s)
I20260808 04:45:29.342237 26871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:29.345269 26979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:29.349879 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 26871
I20260808 04:45:29.349974 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:29.350234 22538 external_mini_cluster.cc:1428] Setting key 888e3a5b6dfcb4053b708a14a34085cb
I20260808 04:45:29.351868 26979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3: Bootstrap starting.
I20260808 04:45:29.354974 26979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:29.355981 26979 log.cc:824] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:29.357897 26979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3: No bootstrap required, opened a new log
Aug 08 04:45:29 dist-test-slave-04bb krb5kdc[26851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:29.361495 26979 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b8b09904644befb0608c78b592a9f3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34029 } }
I20260808 04:45:29.361784 26979 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:29.361903 26979 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b8b09904644befb0608c78b592a9f3, State: Initialized, Role: FOLLOWER
I20260808 04:45:29.362423 26979 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b8b09904644befb0608c78b592a9f3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34029 } }
I20260808 04:45:29.362614 26979 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:29.362696 26979 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:29.362815 26979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:29.363888 26979 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b8b09904644befb0608c78b592a9f3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34029 } }
I20260808 04:45:29.364288 26979 leader_election.cc:302] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7b8b09904644befb0608c78b592a9f3; no voters:
I20260808 04:45:29.364648 26979 leader_election.cc:288] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:29.364755 26985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:29.364971 26985 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: e7b8b09904644befb0608c78b592a9f3, State: Running, Role: LEADER
I20260808 04:45:29.365324 26985 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b8b09904644befb0608c78b592a9f3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34029 } }
I20260808 04:45:29.365991 26979 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:29.367041 26987 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7b8b09904644befb0608c78b592a9f3. Latest consensus state: current_term: 1 leader_uuid: "e7b8b09904644befb0608c78b592a9f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b8b09904644befb0608c78b592a9f3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34029 } } }
I20260808 04:45:29.367194 26987 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:29.367388 26983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:29.351605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60704 (local address 127.22.2.190:34029)
0808 04:45:29.352106 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:29.352132 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:29.352183 (+ 51us) server_negotiation.cc:407] Connection header received
0808 04:45:29.353205 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:29.353233 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:29.353689 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:29.354086 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:29.354779 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:29.356493 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:29.357520 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:29.358113 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:29.358319 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:29.361304 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:29.361311 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:29.361327 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:29.361358 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:29.363531 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:29.363983 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:29.363990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:29.363997 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:29.364137 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:29.364432 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:29.364436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:29.364437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:29.364984 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:29.365863 (+ 879us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:29.366028 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 04:45:29.366347 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":111,"threads_started":1}
I20260808 04:45:29.367476 26986 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7b8b09904644befb0608c78b592a9f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b8b09904644befb0608c78b592a9f3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34029 } } }
I20260808 04:45:29.367556 26986 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:29.367583 27002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:29.370814 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:45:29.371089 27002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:29.377156 27002 catalog_manager.cc:1347] Generated new cluster ID: c0236efdd2b941f2be9d86f41ac72c40
I20260808 04:45:29.377233 27002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:29.384068 27002 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:29.385281 27002 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:29.390798 27002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7b8b09904644befb0608c78b592a9f3: Generated new TSK 0
I20260808 04:45:29.391501 27002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:45:29.392809 26983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:29.387009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.22.2.190:34029)
0808 04:45:29.387111 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:29.387115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:29.387130 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:29.387239 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:29.387243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:29.387288 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:29.387368 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:29.388041 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:29.388865 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:29.390199 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:29.390574 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:29.390700 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:29.391824 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:29.391828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:29.391831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:29.392333 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:29.392361 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:29.392379 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:29.392451 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:29.392624 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 04:45:29.392692 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 04:45:29.398134 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 26871
2026-08-08T04:45:29Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1622 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:29 dist-test-slave-04bb krb5kdc[27034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:29 dist-test-slave-04bb krb5kdc[27034](info): set up 2 sockets
Aug 08 04:45:29 dist-test-slave-04bb krb5kdc[27034](info): commencing operation
krb5kdc: starting...
W20260808 04:45:30.637518 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:30 dist-test-slave-04bb krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:30.766897 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41761
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:30.881158 27052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:30.881459 27052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:30.881529 27052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:30.885546 27052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:30.885627 27052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:30.885666 27052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:30.885717 27052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:30.885758 27052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:30.890990 27052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:30.892043 27052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:30.893153 27052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:30.901801 27063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:30.901810 27060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:30.901888 27059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:30.901989 27052 server_base.cc:1034] running on GCE node
I20260808 04:45:30.902740 27052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:30.903882 27052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:30.905046 27052 hybrid_clock.cc:648] HybridClock initialized: now 1786164330905004 us; error 55 us; skew 500 ppm
Aug 08 04:45:30 dist-test-slave-04bb krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164330, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:30.907950 27052 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:30.908932 27052 webserver.cc:460] Webserver started at http://127.22.2.190:37861/ using document root <none> and password file <none>
I20260808 04:45:30.909555 27052 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:30.909647 27052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:30.909888 27052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:30.911953 27052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "977d42e03d9148309e64e936f048fcfd"
format_stamp: "Formatted at 2026-08-08 04:45:30 on dist-test-slave-04bb"
server_key: "7a7b8a2784317359cb4fce862b3fec92"
server_key_iv: "f3f3f9156b66cf92b1b8cb7154695dc0"
server_key_version: "encryptionkey@0"
I20260808 04:45:30.912595 27052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "977d42e03d9148309e64e936f048fcfd"
format_stamp: "Formatted at 2026-08-08 04:45:30 on dist-test-slave-04bb"
server_key: "7a7b8a2784317359cb4fce862b3fec92"
server_key_iv: "f3f3f9156b66cf92b1b8cb7154695dc0"
server_key_version: "encryptionkey@0"
I20260808 04:45:30.916357 27052 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:30.918803 27077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:30.920010 27052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:30.920146 27052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "977d42e03d9148309e64e936f048fcfd"
format_stamp: "Formatted at 2026-08-08 04:45:30 on dist-test-slave-04bb"
server_key: "7a7b8a2784317359cb4fce862b3fec92"
server_key_iv: "f3f3f9156b66cf92b1b8cb7154695dc0"
server_key_version: "encryptionkey@0"
I20260808 04:45:30.920254 27052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:30.938902 27052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:30.942313 27052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:30.942528 27052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:30.950374 27052 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41761
I20260808 04:45:30.950649 27156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41761 every 8 connection(s)
I20260808 04:45:30.951409 27052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:30.953307 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 27052
I20260808 04:45:30.953404 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:30.953699 22538 external_mini_cluster.cc:1428] Setting key 5051a00dae1b5973e165e4ac0115c6b8
I20260808 04:45:30.954967 27158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:30.962584 27158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd: Bootstrap starting.
Aug 08 04:45:30 dist-test-slave-04bb krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:30.965333 27158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:30.966153 27158 log.cc:824] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:30.968307 27158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd: No bootstrap required, opened a new log
I20260808 04:45:30.969013 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:30.955214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34108 (local address 127.22.2.190:41761)
0808 04:45:30.955615 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:30.955631 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:30.955668 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:45:30.956293 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:30.956328 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:30.956713 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:30.956993 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:30.958077 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:30.959062 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:30.960375 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:30.960903 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:30.961025 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:30.964049 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:30.964054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:30.964068 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:30.964107 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:30.966014 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:30.966548 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:30.966553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:30.966557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:30.966679 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:30.966970 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:30.966974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:30.966976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:30.967369 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:30.967577 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:30.967908 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 04:45:30.968202 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":125,"threads_started":1}
I20260808 04:45:30.972298 27158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977d42e03d9148309e64e936f048fcfd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41761 } }
I20260808 04:45:30.972566 27158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:30.972651 27158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977d42e03d9148309e64e936f048fcfd, State: Initialized, Role: FOLLOWER
I20260808 04:45:30.973106 27158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977d42e03d9148309e64e936f048fcfd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41761 } }
I20260808 04:45:30.973269 27158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:30.973361 27158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:30.973487 27158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:30.974434 27158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977d42e03d9148309e64e936f048fcfd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41761 } }
I20260808 04:45:30.974810 27158 leader_election.cc:302] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 977d42e03d9148309e64e936f048fcfd; no voters:
I20260808 04:45:30.975132 27158 leader_election.cc:288] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:30.975276 27168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:30.975533 27168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [term 1 LEADER]: Becoming Leader. State: Replica: 977d42e03d9148309e64e936f048fcfd, State: Running, Role: LEADER
I20260808 04:45:30.975888 27168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977d42e03d9148309e64e936f048fcfd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41761 } }
I20260808 04:45:30.976466 27158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:30.977499 27171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 977d42e03d9148309e64e936f048fcfd. Latest consensus state: current_term: 1 leader_uuid: "977d42e03d9148309e64e936f048fcfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977d42e03d9148309e64e936f048fcfd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41761 } } }
I20260808 04:45:30.977630 27171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:30.977968 27170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "977d42e03d9148309e64e936f048fcfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977d42e03d9148309e64e936f048fcfd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41761 } } }
I20260808 04:45:30.978046 27170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:30.978374 27181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:30.981050 27181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:30.985946 27181 catalog_manager.cc:1347] Generated new cluster ID: 51fd89bc6646445999efbefcbd29fdc0
I20260808 04:45:30.986009 27181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:30.997017 27181 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:30.997937 27181 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:31.007597 27181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 977d42e03d9148309e64e936f048fcfd: Generated new TSK 0
I20260808 04:45:31.008301 27181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:31.043874 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41761
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:31.159006 27198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:31.159314 27198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:31.159392 27198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:31.163508 27198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:31.163597 27198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:31.163689 27198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:31.169068 27198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:31.170138 27198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:31.171185 27198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:31.181255 27207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:31.181268 27208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:31.181455 27198 server_base.cc:1034] running on GCE node
W20260808 04:45:31.181638 27211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:31.182024 27198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:31.182682 27198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:31.183856 27198 hybrid_clock.cc:648] HybridClock initialized: now 1786164331183840 us; error 24 us; skew 500 ppm
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164331, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:31.186834 27198 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:31.187623 27198 webserver.cc:460] Webserver started at http://127.22.2.129:43065/ using document root <none> and password file <none>
I20260808 04:45:31.188156 27198 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:31.188215 27198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:31.188382 27198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:31.190398 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "2ff197c8ac4346a382840b236920bd4c"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "6ebc58ea5cd03586adc3a2f75df882d9"
server_key_iv: "adc23bba7cb66555a72ad7ffe721967b"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.190917 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "2ff197c8ac4346a382840b236920bd4c"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "6ebc58ea5cd03586adc3a2f75df882d9"
server_key_iv: "adc23bba7cb66555a72ad7ffe721967b"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.194825 27198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:45:31.197382 27219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.198654 27198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:31.198776 27198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "2ff197c8ac4346a382840b236920bd4c"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "6ebc58ea5cd03586adc3a2f75df882d9"
server_key_iv: "adc23bba7cb66555a72ad7ffe721967b"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.198894 27198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:31.215600 27198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:31.219064 27198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:31.219290 27198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:31.219940 27198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:31.220943 27198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:31.220995 27198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.221063 27198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:31.221098 27198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.230943 27198 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:46273
I20260808 04:45:31.230957 27379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:46273 every 8 connection(s)
I20260808 04:45:31.231856 27198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:31.241178 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 27198
I20260808 04:45:31.241286 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:31.241575 22538 external_mini_cluster.cc:1428] Setting key 449672c076fa1fac87e988dd77d2a8f3
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164331, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:31.247213 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:31.233759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:49653 (local address 127.22.2.190:41761)
0808 04:45:31.233931 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:31.233936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:31.234798 (+ 862us) server_negotiation.cc:407] Connection header received
0808 04:45:31.236429 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:31.236433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:31.236499 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:31.236650 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:31.237928 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.238528 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.239802 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.240071 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.240198 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:31.243040 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:31.243048 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:31.243053 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:31.243087 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:31.244816 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:31.245508 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:31.245511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:31.245513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:31.245599 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:31.246036 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:31.246038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:31.246039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:31.246219 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:31.246305 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:31.246986 (+ 681us) server_negotiation.cc:299] Negotiation successful
0808 04:45:31.247092 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 04:45:31.248651 27383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:31.234143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41761 (local address 127.22.2.129:49653)
0808 04:45:31.234633 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:31.234682 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:31.236184 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:31.236836 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:31.236843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:31.237241 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:31.237751 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:31.237766 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.238670 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.238676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:31.239653 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:31.239663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.240245 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.240249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:31.240435 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:31.241118 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:31.241144 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:31.242873 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:31.244947 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:31.244954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:31.244970 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:31.245394 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:31.245709 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:31.245715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:31.245719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:31.245926 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:31.246333 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:31.246342 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:31.246698 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 04:45:31.246961 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 04:45:31.247372 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":103,"threads_started":1}
I20260808 04:45:31.249910 27380 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41761
I20260808 04:45:31.250223 27380 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:31.250910 27380 heartbeater.cc:511] Master 127.22.2.190:41761 requested a full tablet report, sending...
I20260808 04:45:31.252627 27107 ts_manager.cc:194] Registered new tserver with Master: 2ff197c8ac4346a382840b236920bd4c (127.22.2.129:46273)
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:45:31.253800 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:49653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:31.273078 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41761
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:31.386752 27386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:31.386988 27386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:31.387029 27386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:31.391065 27386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:31.391124 27386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:31.391183 27386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:31.396263 27386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:31.397300 27386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:31.398322 27386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:31.407877 27398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:31.407927 27393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:31.407877 27392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:31.408175 27386 server_base.cc:1034] running on GCE node
I20260808 04:45:31.408638 27386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:31.409224 27386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:31.410414 27386 hybrid_clock.cc:648] HybridClock initialized: now 1786164331410397 us; error 28 us; skew 500 ppm
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164331, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:31.413177 27386 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:31.414064 27386 webserver.cc:460] Webserver started at http://127.22.2.130:44715/ using document root <none> and password file <none>
I20260808 04:45:31.414633 27386 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:31.414712 27386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:31.414927 27386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:31.416989 27386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "0399d0cced7a4fd389ad50f0f7e5629c"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "9d68059f7b523f7d04c7ac86307db38d"
server_key_iv: "d1a9f994eaaf9f04be3f7725fae889c6"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.417549 27386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "0399d0cced7a4fd389ad50f0f7e5629c"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "9d68059f7b523f7d04c7ac86307db38d"
server_key_iv: "d1a9f994eaaf9f04be3f7725fae889c6"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.421270 27386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:45:31.423678 27405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.425031 27386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 04:45:31.425144 27386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "0399d0cced7a4fd389ad50f0f7e5629c"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "9d68059f7b523f7d04c7ac86307db38d"
server_key_iv: "d1a9f994eaaf9f04be3f7725fae889c6"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.425269 27386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:31.439420 27386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:31.442397 27386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:31.442687 27386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:31.443365 27386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:31.444262 27386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:31.444314 27386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.444388 27386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:31.444429 27386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.454003 27386 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35843
I20260808 04:45:31.454030 27582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35843 every 8 connection(s)
I20260808 04:45:31.454869 27386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:31.460150 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 27386
I20260808 04:45:31.460282 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:31.460646 22538 external_mini_cluster.cc:1428] Setting key b7422fb5517815572eed86ac1a5799a7
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164331, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:31.469559 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:31.456678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:54053 (local address 127.22.2.190:41761)
0808 04:45:31.456808 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:31.456813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:31.457617 (+ 804us) server_negotiation.cc:407] Connection header received
0808 04:45:31.458999 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:31.459002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:31.459063 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:31.459166 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:31.460579 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.461268 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.462716 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.462937 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.463019 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:31.465807 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:31.465811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:31.465813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:31.465837 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:31.467348 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:31.467989 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:31.467992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:31.467993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:31.468067 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:31.468427 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:31.468431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:31.468432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:31.468668 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:31.468777 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:31.469263 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 04:45:31.469389 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 04:45:31.470350 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:31.457027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41761 (local address 127.22.2.130:54053)
0808 04:45:31.457470 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:31.457512 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:31.458784 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:31.459350 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:31.459357 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:31.459698 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:31.460359 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:31.460381 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.461396 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.461400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:31.462583 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:31.462599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.463089 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.463095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:31.463332 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:31.463986 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:31.464012 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:31.465618 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:31.467466 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:31.467472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:31.467487 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:31.467883 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:31.468164 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:31.468168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:31.468170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:31.468324 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:31.468779 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:31.468788 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:31.469023 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 04:45:31.469210 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 04:45:31.469538 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:45:31.471632 27583 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41761
I20260808 04:45:31.471890 27583 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:31.472496 27583 heartbeater.cc:511] Master 127.22.2.190:41761 requested a full tablet report, sending...
I20260808 04:45:31.473583 27107 ts_manager.cc:194] Registered new tserver with Master: 0399d0cced7a4fd389ad50f0f7e5629c (127.22.2.130:35843)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:45:31.474335 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:54053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:31.489144 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41761
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:31.603713 27591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:31.603971 27591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:31.604010 27591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:31.607995 27591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:31.608047 27591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:31.608139 27591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:31.613281 27591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:31.614303 27591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:31.615347 27591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:31.624975 27601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:31.624972 27602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:31.625151 27591 server_base.cc:1034] running on GCE node
W20260808 04:45:31.624972 27604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:31.625658 27591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:31.626284 27591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:31.627453 27591 hybrid_clock.cc:648] HybridClock initialized: now 1786164331627437 us; error 26 us; skew 500 ppm
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164331, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:31.630298 27591 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:31.631165 27591 webserver.cc:460] Webserver started at http://127.22.2.131:45633/ using document root <none> and password file <none>
I20260808 04:45:31.631718 27591 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:31.631805 27591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:31.632026 27591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:31.634037 27591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "d9e6844d00184a4cb4390409dc131b67"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "52adee52fefa6ba29a3a51c04548d768"
server_key_iv: "56320ede586c23a9dabb94b1eb9133ae"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.634599 27591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "d9e6844d00184a4cb4390409dc131b67"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "52adee52fefa6ba29a3a51c04548d768"
server_key_iv: "56320ede586c23a9dabb94b1eb9133ae"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.638494 27591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 04:45:31.640933 27616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.642107 27591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:31.642238 27591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "d9e6844d00184a4cb4390409dc131b67"
format_stamp: "Formatted at 2026-08-08 04:45:31 on dist-test-slave-04bb"
server_key: "52adee52fefa6ba29a3a51c04548d768"
server_key_iv: "56320ede586c23a9dabb94b1eb9133ae"
server_key_version: "encryptionkey@0"
I20260808 04:45:31.642339 27591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:31.655427 27591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:31.658232 27591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:31.658422 27591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:31.658975 27591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:31.659787 27591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:31.659862 27591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.659929 27591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:31.659979 27591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:31.668469 27591 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:43333
I20260808 04:45:31.668493 27790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:43333 every 8 connection(s)
I20260808 04:45:31.669364 27591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:31.675894 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 27591
I20260808 04:45:31.676018 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:31.676357 22538 external_mini_cluster.cc:1428] Setting key 7887c478d4d04188b0107bea6f62fd42
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164331, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:31.682969 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:31.671199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35411 (local address 127.22.2.190:41761)
0808 04:45:31.671355 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:31.671360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:31.672148 (+ 788us) server_negotiation.cc:407] Connection header received
0808 04:45:31.673587 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:31.673589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:31.673638 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:31.673704 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:31.674831 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.675351 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.676922 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.677223 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.677348 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:31.679898 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:31.679901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:31.679904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:31.679928 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:31.681248 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:31.681716 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:31.681718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:31.681720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:31.681794 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:31.682078 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:31.682082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:31.682083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:31.682248 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:31.682322 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:31.682754 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 04:45:31.682851 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:31.683714 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:31.671565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41761 (local address 127.22.2.131:35411)
0808 04:45:31.672003 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:31.672044 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:31.673377 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:31.673860 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:31.673867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:31.674251 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:31.674708 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:31.674721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.675452 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.675454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:31.676801 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:31.676811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.677380 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.677384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:31.677560 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:31.678137 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:31.678159 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:31.679674 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:31.681359 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:31.681364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:31.681375 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:31.681626 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:31.681870 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:31.681872 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:31.681874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:31.682001 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:31.682340 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:31.682345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:31.682557 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 04:45:31.682707 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 04:45:31.682938 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260808 04:45:31.684789 27793 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41761
I20260808 04:45:31.685004 27793 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:31.685434 27793 heartbeater.cc:511] Master 127.22.2.190:41761 requested a full tablet report, sending...
I20260808 04:45:31.686385 27107 ts_manager.cc:194] Registered new tserver with Master: d9e6844d00184a4cb4390409dc131b67 (127.22.2.131:43333)
I20260808 04:45:31.686919 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:35411
I20260808 04:45:31.690049 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164331, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164331, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:31.717975 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:31.709071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.22.2.190:41761)
0808 04:45:31.709291 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:31.709297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:31.709453 (+ 156us) server_negotiation.cc:407] Connection header received
0808 04:45:31.709634 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:31.709638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:31.709691 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:31.709780 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:31.710552 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.711085 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.711963 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:31.712189 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:31.712295 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:31.714846 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:31.714849 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:31.714852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:31.714876 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:31.716197 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:31.716691 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:31.716695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:31.716696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:31.716929 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:31.717205 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:31.717208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:31.717209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:31.717428 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:31.717536 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:31.717750 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 04:45:31.717845 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260808 04:45:31.718416 27107 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34120
I20260808 04:45:31.719607 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 27198
I20260808 04:45:31.745801 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 27386
I20260808 04:45:31.756193 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 27591
I20260808 04:45:31.770004 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 27052
2026-08-08T04:45:31Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2369 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27812](info): set up 2 sockets
Aug 08 04:45:31 dist-test-slave-04bb krb5kdc[27812](info): commencing operation
krb5kdc: starting...
W20260808 04:45:33.048810 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:33 dist-test-slave-04bb krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:33.176445 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41243
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:33.291700 27827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:33.291976 27827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:33.292053 27827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:33.296027 27827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:33.296104 27827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:33.296144 27827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:33.296200 27827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:33.296241 27827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:33.301409 27827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:33.302416 27827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:33.303399 27827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:33.311947 27839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:33.311973 27835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:33.311947 27837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:33.312232 27827 server_base.cc:1034] running on GCE node
I20260808 04:45:33.312996 27827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:33.314131 27827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:33.315354 27827 hybrid_clock.cc:648] HybridClock initialized: now 1786164333315339 us; error 27 us; skew 500 ppm
Aug 08 04:45:33 dist-test-slave-04bb krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164333, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:33.318284 27827 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:33.319185 27827 webserver.cc:460] Webserver started at http://127.22.2.190:46427/ using document root <none> and password file <none>
I20260808 04:45:33.319710 27827 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:33.319772 27827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:33.319967 27827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:33.321996 27827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "86acfecab4634c6996c1ccc074d832c2"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "5d516f1902abd34d8a8f272e3b65ea3b"
server_key_iv: "f4f3f6f2b243cc652437bb682ec918c9"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.322570 27827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "86acfecab4634c6996c1ccc074d832c2"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "5d516f1902abd34d8a8f272e3b65ea3b"
server_key_iv: "f4f3f6f2b243cc652437bb682ec918c9"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.326349 27827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:33.328753 27849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:33.329952 27827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:33.330097 27827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "86acfecab4634c6996c1ccc074d832c2"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "5d516f1902abd34d8a8f272e3b65ea3b"
server_key_iv: "f4f3f6f2b243cc652437bb682ec918c9"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.330186 27827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:33.346632 27827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:33.349514 27827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:33.349705 27827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:33.356420 27827 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41243
I20260808 04:45:33.356475 27929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41243 every 8 connection(s)
I20260808 04:45:33.357429 27827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:33.360190 27930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:33.363157 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 27827
I20260808 04:45:33.363298 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:33.363587 22538 external_mini_cluster.cc:1428] Setting key 777b45332881f967a0a50d04114fc011
I20260808 04:45:33.366619 27930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2: Bootstrap starting.
I20260808 04:45:33.369429 27930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:33.370249 27930 log.cc:824] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:33.372323 27930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2: No bootstrap required, opened a new log
Aug 08 04:45:33 dist-test-slave-04bb krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:33.376735 27930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86acfecab4634c6996c1ccc074d832c2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41243 } }
I20260808 04:45:33.376988 27930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:33.377063 27930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86acfecab4634c6996c1ccc074d832c2, State: Initialized, Role: FOLLOWER
I20260808 04:45:33.377561 27930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86acfecab4634c6996c1ccc074d832c2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41243 } }
I20260808 04:45:33.377724 27930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:33.377800 27930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:33.377897 27930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:33.379011 27930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86acfecab4634c6996c1ccc074d832c2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41243 } }
I20260808 04:45:33.379386 27930 leader_election.cc:302] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86acfecab4634c6996c1ccc074d832c2; no voters:
I20260808 04:45:33.379680 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:33.364987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.22.2.190:41243)
0808 04:45:33.365412 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:33.365431 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:33.365474 (+ 43us) server_negotiation.cc:407] Connection header received
0808 04:45:33.366248 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:33.366284 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:33.366696 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:33.367056 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:33.367740 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.369249 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.370155 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.370820 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.370948 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:33.373662 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:33.373668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:33.373685 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:33.373716 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:33.376558 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:33.376992 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:33.377000 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:33.377009 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:33.377158 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:33.377407 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:33.377410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:33.377411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:33.377936 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:33.378141 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:33.378363 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 04:45:33.378655 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":106,"threads_started":1}
I20260808 04:45:33.379731 27930 leader_election.cc:288] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:33.379876 27937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:33.380254 27937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [term 1 LEADER]: Becoming Leader. State: Replica: 86acfecab4634c6996c1ccc074d832c2, State: Running, Role: LEADER
I20260808 04:45:33.380610 27937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86acfecab4634c6996c1ccc074d832c2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41243 } }
I20260808 04:45:33.381155 27930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:33.381934 27938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86acfecab4634c6996c1ccc074d832c2. Latest consensus state: current_term: 1 leader_uuid: "86acfecab4634c6996c1ccc074d832c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86acfecab4634c6996c1ccc074d832c2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41243 } } }
I20260808 04:45:33.382084 27938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:33.382550 27955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:33.382843 27940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86acfecab4634c6996c1ccc074d832c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86acfecab4634c6996c1ccc074d832c2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41243 } } }
I20260808 04:45:33.382946 27940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:33.386557 27955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:33.392009 27955 catalog_manager.cc:1347] Generated new cluster ID: 4167101a23a549aa949c95c793ea3485
I20260808 04:45:33.392089 27955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:33.414762 27955 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:33.416070 27955 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:33.424350 27955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86acfecab4634c6996c1ccc074d832c2: Generated new TSK 0
I20260808 04:45:33.425150 27955 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:33.455608 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41243
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:33.569177 27975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:33.569459 27975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:33.569540 27975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:33.573551 27975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:33.573632 27975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:33.573721 27975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:33.578891 27975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:33.579945 27975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:33.581039 27975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:33.590998 27982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:33.590983 27984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:33.590969 27986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:33.591256 27975 server_base.cc:1034] running on GCE node
I20260808 04:45:33.591694 27975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:33.592331 27975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:33.593586 27975 hybrid_clock.cc:648] HybridClock initialized: now 1786164333593554 us; error 39 us; skew 500 ppm
Aug 08 04:45:33 dist-test-slave-04bb krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164333, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:33.596689 27975 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:33.597647 27975 webserver.cc:460] Webserver started at http://127.22.2.129:41617/ using document root <none> and password file <none>
I20260808 04:45:33.598254 27975 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:33.598361 27975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:33.598613 27975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:33.600951 27975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "734f905eb76c47dead1cd2aea3e62e97"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "cf69c5b08814d918bdc855202062b05c"
server_key_iv: "e89338fae7f541515fab3c52671cc002"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.601601 27975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "734f905eb76c47dead1cd2aea3e62e97"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "cf69c5b08814d918bdc855202062b05c"
server_key_iv: "e89338fae7f541515fab3c52671cc002"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.605597 27975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:45:33.608121 27997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:33.609428 27975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:33.609586 27975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "734f905eb76c47dead1cd2aea3e62e97"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "cf69c5b08814d918bdc855202062b05c"
server_key_iv: "e89338fae7f541515fab3c52671cc002"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.609712 27975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:33.628034 27975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:33.631033 27975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:33.631253 27975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:33.631875 27975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:33.632850 27975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:33.632923 27975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:33.633066 27975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:33.633127 27975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:33.642594 27975 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33263
I20260808 04:45:33.642642 28169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33263 every 8 connection(s)
I20260808 04:45:33.643520 27975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:33.652585 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 27975
I20260808 04:45:33.652745 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:33.653057 22538 external_mini_cluster.cc:1428] Setting key e543ef9aa23ef33297e27f0a0a489a76
Aug 08 04:45:33 dist-test-slave-04bb krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164333, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:33.657884 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:33.645426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:43279 (local address 127.22.2.190:41243)
0808 04:45:33.645559 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:33.645563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:33.646425 (+ 862us) server_negotiation.cc:407] Connection header received
0808 04:45:33.647833 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:33.647837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:33.647896 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:33.648000 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:33.649195 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.649732 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.650843 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.651074 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.651181 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:33.653923 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:33.653926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:33.653928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:33.653952 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:33.655689 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:33.656171 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:33.656175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:33.656178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:33.656275 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:33.656628 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:33.656635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:33.656638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:33.656900 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:33.656981 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:33.657654 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 04:45:33.657766 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 04:45:33.658798 28173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:33.645787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41243 (local address 127.22.2.129:43279)
0808 04:45:33.646282 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:33.646323 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:33.647619 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:33.648188 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:33.648195 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:33.648592 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:33.649050 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:33.649062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.649855 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.649859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:33.650718 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:33.650727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.651242 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.651247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:33.651403 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:33.651933 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:33.651948 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:33.653745 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:33.655810 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:33.655815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:33.655826 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:33.656074 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:33.656370 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:33.656372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:33.656374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:33.656503 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:33.657014 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:33.657022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:33.657367 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 04:45:33.657582 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 04:45:33.657924 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":107,"threads_started":1}
I20260808 04:45:33.660032 28170 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41243
I20260808 04:45:33.660298 28170 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:33.660920 28170 heartbeater.cc:511] Master 127.22.2.190:41243 requested a full tablet report, sending...
I20260808 04:45:33.662503 27874 ts_manager.cc:194] Registered new tserver with Master: 734f905eb76c47dead1cd2aea3e62e97 (127.22.2.129:33263)
I20260808 04:45:33.663601 27874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:43279
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:33.683105 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41243
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:33.799448 28177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:33.799706 28177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:33.799750 28177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:33.803653 28177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:33.803709 28177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:33.803766 28177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:33.808923 28177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:33.809959 28177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:33.810922 28177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:33.820699 28189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:33.820734 28186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:33.820734 28185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:33.821419 28177 server_base.cc:1034] running on GCE node
I20260808 04:45:33.821837 28177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:33.822487 28177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:33.823668 28177 hybrid_clock.cc:648] HybridClock initialized: now 1786164333823635 us; error 47 us; skew 500 ppm
Aug 08 04:45:33 dist-test-slave-04bb krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164333, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:33.826406 28177 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:33.827255 28177 webserver.cc:460] Webserver started at http://127.22.2.130:33281/ using document root <none> and password file <none>
I20260808 04:45:33.827836 28177 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:33.827925 28177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:33.828171 28177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:33.830224 28177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "122daee95d9548b59f9ab63518c3d0ea"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "c72774336362a91ed40a6230600cf50b"
server_key_iv: "74ef468154eb11888227a90c288fbef2"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.830782 28177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "122daee95d9548b59f9ab63518c3d0ea"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "c72774336362a91ed40a6230600cf50b"
server_key_iv: "74ef468154eb11888227a90c288fbef2"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.834534 28177 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:33.836998 28200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:33.838287 28177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:33.838430 28177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "122daee95d9548b59f9ab63518c3d0ea"
format_stamp: "Formatted at 2026-08-08 04:45:33 on dist-test-slave-04bb"
server_key: "c72774336362a91ed40a6230600cf50b"
server_key_iv: "74ef468154eb11888227a90c288fbef2"
server_key_version: "encryptionkey@0"
I20260808 04:45:33.838536 28177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:33.861344 28177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:33.864308 28177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:33.864513 28177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:33.865139 28177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:33.865972 28177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:33.866041 28177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:33.866127 28177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:33.866173 28177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:33.875211 28177 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:43367
I20260808 04:45:33.875232 28373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:43367 every 8 connection(s)
I20260808 04:45:33.876112 28177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:33.879655 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 28177
I20260808 04:45:33.879769 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:33.880033 22538 external_mini_cluster.cc:1428] Setting key ed0d5e1949488334fe20481a4a26df21
Aug 08 04:45:33 dist-test-slave-04bb krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164333, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:45:33.891750 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:33.877988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:41397 (local address 127.22.2.190:41243)
0808 04:45:33.878133 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:33.878138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:33.879119 (+ 981us) server_negotiation.cc:407] Connection header received
0808 04:45:33.880665 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:33.880668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:33.880724 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:33.880819 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:33.882439 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.883274 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.884974 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.885189 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.885302 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:33.888124 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:33.888128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:33.888131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:33.888159 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:33.889682 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:33.890194 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:33.890198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:33.890200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:33.890289 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:33.890632 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:33.890637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:33.890639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:33.890845 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:33.890951 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:33.891443 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 04:45:33.891569 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:45:33.892648 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:33.878417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41243 (local address 127.22.2.130:41397)
0808 04:45:33.878966 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:33.879010 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:33.880409 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:33.881037 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:33.881071 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:33.881597 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:33.882231 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:33.882251 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.883402 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.883405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:33.884848 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:33.884862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:33.885361 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:33.885365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:33.885535 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:33.886147 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:33.886163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:33.887951 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:33.889803 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:33.889809 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:33.889825 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:33.890084 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:33.890386 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:33.890390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:33.890392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:33.890537 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:33.890952 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:33.890960 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:33.891209 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 04:45:33.891400 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 04:45:33.891745 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:45:33.893941 28375 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41243
I20260808 04:45:33.894198 28375 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:33.894703 28375 heartbeater.cc:511] Master 127.22.2.190:41243 requested a full tablet report, sending...
I20260808 04:45:33.895884 27874 ts_manager.cc:194] Registered new tserver with Master: 122daee95d9548b59f9ab63518c3d0ea (127.22.2.130:43367)
I20260808 04:45:33.896510 27874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:41397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:33.909821 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41243
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:34.023790 28384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:34.024073 28384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:34.024165 28384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:34.028179 28384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:34.028260 28384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:34.028347 28384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:34.033600 28384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:34.034673 28384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:34.035684 28384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:34.045717 28389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:34.045724 28392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:34.045717 28396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:34.045991 28384 server_base.cc:1034] running on GCE node
I20260808 04:45:34.046466 28384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:34.047113 28384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:34.048307 28384 hybrid_clock.cc:648] HybridClock initialized: now 1786164334048278 us; error 41 us; skew 500 ppm
Aug 08 04:45:34 dist-test-slave-04bb krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164334, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:34.051182 28384 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:34.052031 28384 webserver.cc:460] Webserver started at http://127.22.2.131:36089/ using document root <none> and password file <none>
I20260808 04:45:34.052652 28384 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:34.052750 28384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:34.052976 28384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:34.054973 28384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "a2be8101c3f14107b2e0964ea2818a19"
format_stamp: "Formatted at 2026-08-08 04:45:34 on dist-test-slave-04bb"
server_key: "f6e1db11260ef3e455b965e109510f6b"
server_key_iv: "5698431263fff00b84ff0dce9275169e"
server_key_version: "encryptionkey@0"
I20260808 04:45:34.055537 28384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "a2be8101c3f14107b2e0964ea2818a19"
format_stamp: "Formatted at 2026-08-08 04:45:34 on dist-test-slave-04bb"
server_key: "f6e1db11260ef3e455b965e109510f6b"
server_key_iv: "5698431263fff00b84ff0dce9275169e"
server_key_version: "encryptionkey@0"
I20260808 04:45:34.059242 28384 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:45:34.061769 28410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:34.062950 28384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:34.063081 28384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "a2be8101c3f14107b2e0964ea2818a19"
format_stamp: "Formatted at 2026-08-08 04:45:34 on dist-test-slave-04bb"
server_key: "f6e1db11260ef3e455b965e109510f6b"
server_key_iv: "5698431263fff00b84ff0dce9275169e"
server_key_version: "encryptionkey@0"
I20260808 04:45:34.063185 28384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:34.076203 28384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:34.079350 28384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:34.079535 28384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:34.080088 28384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:34.080989 28384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:34.081067 28384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:34.081152 28384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:34.081197 28384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:34.090536 28384 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:38509
I20260808 04:45:34.090593 28598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:38509 every 8 connection(s)
I20260808 04:45:34.091431 28384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:34.096839 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 28384
I20260808 04:45:34.096978 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:34.097355 22538 external_mini_cluster.cc:1428] Setting key dccbf13b0c24d9ce7f934fcb237b2541
Aug 08 04:45:34 dist-test-slave-04bb krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164334, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:34.107074 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:34.093767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:58893 (local address 127.22.2.190:41243)
0808 04:45:34.093984 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:34.093988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:34.095723 (+ 1735us) server_negotiation.cc:407] Connection header received
0808 04:45:34.097635 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:34.097638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:34.097699 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:34.097779 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:34.099217 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:34.099746 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:34.100944 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:34.101172 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:34.101282 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:34.103583 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:34.103587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:34.103590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:34.103620 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:34.105110 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:34.105598 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:34.105601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:34.105603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:34.105684 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:34.106011 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:34.106015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:34.106017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:34.106194 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:34.106301 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:34.106776 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 04:45:34.106905 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 04:45:34.107910 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:34.094197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41243 (local address 127.22.2.131:58893)
0808 04:45:34.095543 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:34.095601 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:34.097415 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:34.097954 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:34.097963 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:34.098456 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:34.099069 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:34.099085 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:34.099878 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:34.099882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:34.100817 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:34.100827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:34.101326 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:34.101330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:34.101497 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:34.102007 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:34.102023 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:34.103425 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:34.105222 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:34.105227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:34.105241 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:34.105492 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:34.105775 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:34.105778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:34.105780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:34.105914 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:34.106302 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:34.106318 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:34.106543 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 04:45:34.106732 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 04:45:34.107035 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1108,"thread_start_us":142,"threads_started":1}
I20260808 04:45:34.109092 28599 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41243
I20260808 04:45:34.109372 28599 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:34.109853 28599 heartbeater.cc:511] Master 127.22.2.190:41243 requested a full tablet report, sending...
I20260808 04:45:34.111121 27874 ts_manager.cc:194] Registered new tserver with Master: a2be8101c3f14107b2e0964ea2818a19 (127.22.2.131:38509)
I20260808 04:45:34.111584 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:45:34.111824 27874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:58893
Aug 08 04:45:34 dist-test-slave-04bb krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:45:34 dist-test-slave-04bb krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:34.138811 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:34.129459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.22.2.129:33263)
0808 04:45:34.129770 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:34.129786 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:34.129812 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:45:34.129867 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:34.129873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:34.130066 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:34.130190 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:34.130859 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:34.131652 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:34.132552 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:34.132933 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:34.133024 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:34.135529 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:34.135539 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:34.135553 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:34.135577 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:34.137175 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:34.137592 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:34.137598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:34.137599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:34.137680 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:34.137952 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:34.137956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:34.137957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:34.138200 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:34.138383 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:34.138532 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 04:45:34.138656 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":99,"threads_started":1}
Aug 08 04:45:34 dist-test-slave-04bb krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:34.149935 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:34.141817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.22.2.190:41243)
0808 04:45:34.141956 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:34.141961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:34.142007 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:45:34.142167 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:34.142170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:34.142219 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:34.142301 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:34.143017 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:34.143565 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:34.144365 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:34.144684 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:34.144794 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:34.147106 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:34.147109 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:34.147111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:34.147134 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:34.148605 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:34.148962 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:34.148965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:34.148966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:34.149037 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:34.149249 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:34.149251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:34.149252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:34.149450 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:34.149554 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:34.149730 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 04:45:34.149821 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 04:45:34.150332 27874 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56868
I20260808 04:45:34.150907 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 27975
I20260808 04:45:34.166807 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 28177
I20260808 04:45:34.177848 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 28384
I20260808 04:45:34.186862 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 27827
2026-08-08T04:45:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2414 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:34 dist-test-slave-04bb krb5kdc[28621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:34 dist-test-slave-04bb krb5kdc[28621](info): set up 2 sockets
Aug 08 04:45:34 dist-test-slave-04bb krb5kdc[28621](info): commencing operation
krb5kdc: starting...
W20260808 04:45:35.410614 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:35 dist-test-slave-04bb krb5kdc[28621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:35.537180 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:39375
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:39375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:35.649438 28638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:35.649719 28638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:35.649789 28638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:35.653827 28638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:35.653915 28638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:35.653949 28638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:35.653987 28638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:35.654019 28638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:35.659175 28638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:39375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:39375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:35.660195 28638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:35.661222 28638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:35.669358 28645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:35.669345 28648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:35.669350 28646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:35.669641 28638 server_base.cc:1034] running on GCE node
I20260808 04:45:35.670255 28638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:35.671329 28638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:35.672559 28638 hybrid_clock.cc:648] HybridClock initialized: now 1786164335672495 us; error 40 us; skew 500 ppm
Aug 08 04:45:35 dist-test-slave-04bb krb5kdc[28621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164335, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:35.675391 28638 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:35.676155 28638 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:35.676220 28638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:35.676472 28638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:35.678469 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "833a2fc56f1f4fbe95ca422fc44258ad"
format_stamp: "Formatted at 2026-08-08 04:45:35 on dist-test-slave-04bb"
server_key: "e3f512731ed7d6ebe8663f94ad3ce4cc"
server_key_iv: "8fdf2af378d683e059b760051407f6ce"
server_key_version: "encryptionkey@0"
I20260808 04:45:35.679026 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "833a2fc56f1f4fbe95ca422fc44258ad"
format_stamp: "Formatted at 2026-08-08 04:45:35 on dist-test-slave-04bb"
server_key: "e3f512731ed7d6ebe8663f94ad3ce4cc"
server_key_iv: "8fdf2af378d683e059b760051407f6ce"
server_key_version: "encryptionkey@0"
I20260808 04:45:35.682711 28638 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:35.685143 28656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:35.686345 28638 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:35.686474 28638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "833a2fc56f1f4fbe95ca422fc44258ad"
format_stamp: "Formatted at 2026-08-08 04:45:35 on dist-test-slave-04bb"
server_key: "e3f512731ed7d6ebe8663f94ad3ce4cc"
server_key_iv: "8fdf2af378d683e059b760051407f6ce"
server_key_version: "encryptionkey@0"
I20260808 04:45:35.686570 28638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:35.706429 28638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:35.709792 28638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:35.710000 28638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:35.717258 28638 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:39375
I20260808 04:45:35.717518 28730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:39375 every 8 connection(s)
I20260808 04:45:35.717958 28638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:35.721681 28733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:35.723384 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 28638
I20260808 04:45:35.723520 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:35.723809 22538 external_mini_cluster.cc:1428] Setting key c9df385934fdfcc1c24c15be8716cee6
I20260808 04:45:35.728294 28733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad: Bootstrap starting.
I20260808 04:45:35.730921 28733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:35.731859 28733 log.cc:824] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:45:35 dist-test-slave-04bb krb5kdc[28621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:35.733940 28733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad: No bootstrap required, opened a new log
I20260808 04:45:35.737601 28733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833a2fc56f1f4fbe95ca422fc44258ad" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39375 } }
I20260808 04:45:35.737857 28733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:35.737946 28733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 833a2fc56f1f4fbe95ca422fc44258ad, State: Initialized, Role: FOLLOWER
I20260808 04:45:35.738371 28733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833a2fc56f1f4fbe95ca422fc44258ad" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39375 } }
I20260808 04:45:35.738515 28733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:35.738541 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:35.725318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48548 (local address 127.22.2.190:39375)
0808 04:45:35.725724 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:35.725743 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:35.725786 (+ 43us) server_negotiation.cc:407] Connection header received
0808 04:45:35.726424 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:35.726441 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:35.726757 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:35.727030 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:35.727979 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:35.729101 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:35.730272 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:35.730719 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:35.730842 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:35.733767 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:35.733773 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:35.733787 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:35.733816 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:35.735556 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:35.736005 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:35.736010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:35.736032 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:35.736155 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:35.736458 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:35.736462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:35.736464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:35.736891 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:35.737100 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:35.737385 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 04:45:35.737662 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":131,"threads_started":1}
I20260808 04:45:35.738595 28733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:35.738708 28733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:35.739738 28733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833a2fc56f1f4fbe95ca422fc44258ad" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39375 } }
I20260808 04:45:35.740079 28733 leader_election.cc:302] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 833a2fc56f1f4fbe95ca422fc44258ad; no voters:
I20260808 04:45:35.740402 28733 leader_election.cc:288] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:35.740483 28739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:35.740720 28739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [term 1 LEADER]: Becoming Leader. State: Replica: 833a2fc56f1f4fbe95ca422fc44258ad, State: Running, Role: LEADER
I20260808 04:45:35.741037 28739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833a2fc56f1f4fbe95ca422fc44258ad" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39375 } }
I20260808 04:45:35.741727 28733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:35.742139 28739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 833a2fc56f1f4fbe95ca422fc44258ad. Latest consensus state: current_term: 1 leader_uuid: "833a2fc56f1f4fbe95ca422fc44258ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833a2fc56f1f4fbe95ca422fc44258ad" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39375 } } }
I20260808 04:45:35.742252 28739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:35.742655 28740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "833a2fc56f1f4fbe95ca422fc44258ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833a2fc56f1f4fbe95ca422fc44258ad" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39375 } } }
I20260808 04:45:35.742805 28740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:35.744001 28744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:35.746753 28744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:35.752050 28744 catalog_manager.cc:1347] Generated new cluster ID: c981197a1efb4936b374adeea79d9b98
I20260808 04:45:35.752115 28744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:35.771551 28744 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:35.772816 28744 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:35.779894 28744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad: Generated new TSK 0
I20260808 04:45:35.780767 28744 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:35.819367 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39375
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:35.934384 28770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:35.934623 28770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:35.934665 28770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:35.938637 28770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:35.938688 28770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:35.938745 28770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:35.943993 28770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:35.944959 28770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:35.946055 28770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:35.955363 28777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:35.955332 28779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:35.955564 28770 server_base.cc:1034] running on GCE node
W20260808 04:45:35.955353 28776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:35.956079 28770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:35.956709 28770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:35.957880 28770 hybrid_clock.cc:648] HybridClock initialized: now 1786164335957864 us; error 25 us; skew 500 ppm
Aug 08 04:45:35 dist-test-slave-04bb krb5kdc[28621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164335, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:35.960640 28770 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:35.961376 28770 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:35.961437 28770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:35.961647 28770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:35.963629 28770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "db98b8b20ddc40838de0e585161909b0"
format_stamp: "Formatted at 2026-08-08 04:45:35 on dist-test-slave-04bb"
server_key: "d7f1cc84c3e03fb06eb19291139ce401"
server_key_iv: "3697497c33ec78e3cffc3b1aa6ce7722"
server_key_version: "encryptionkey@0"
I20260808 04:45:35.964212 28770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "db98b8b20ddc40838de0e585161909b0"
format_stamp: "Formatted at 2026-08-08 04:45:35 on dist-test-slave-04bb"
server_key: "d7f1cc84c3e03fb06eb19291139ce401"
server_key_iv: "3697497c33ec78e3cffc3b1aa6ce7722"
server_key_version: "encryptionkey@0"
I20260808 04:45:35.967792 28770 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:45:35.970132 28790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:35.971300 28770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:35.971431 28770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "db98b8b20ddc40838de0e585161909b0"
format_stamp: "Formatted at 2026-08-08 04:45:35 on dist-test-slave-04bb"
server_key: "d7f1cc84c3e03fb06eb19291139ce401"
server_key_iv: "3697497c33ec78e3cffc3b1aa6ce7722"
server_key_version: "encryptionkey@0"
I20260808 04:45:35.971534 28770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:35.982170 28770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:35.984910 28770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:35.985088 28770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:35.985639 28770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:35.986443 28770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:35.986510 28770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:35.986578 28770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:35.986617 28770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:35.995405 28770 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:40621
I20260808 04:45:35.995472 28960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:40621 every 8 connection(s)
I20260808 04:45:35.995992 28770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:35.996240 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 28770
I20260808 04:45:35.996376 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:35.996726 22538 external_mini_cluster.cc:1428] Setting key fddbe6aee9ca159a449bb8bb39b6ce2b
Aug 08 04:45:36 dist-test-slave-04bb krb5kdc[28621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164335, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 04:45:36.011381 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:35.997835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:35139 (local address 127.22.2.190:39375)
0808 04:45:35.997965 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:35.997970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:35.999364 (+ 1394us) server_negotiation.cc:407] Connection header received
0808 04:45:36.000184 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.000187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.000250 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.000348 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:36.001579 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.002097 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.003920 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.004144 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.004235 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.007334 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:36.007338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:36.007340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:36.007367 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:36.009064 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:36.009674 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:36.009684 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:36.009686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:36.009769 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:36.010150 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:36.010155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:36.010157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:36.010357 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:36.010474 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.011020 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.011189 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:45:36.012310 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:35.998276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:39375 (local address 127.22.2.129:35139)
0808 04:45:35.999190 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:35.999243 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:35.999975 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:36.000565 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:36.000577 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:36.000936 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:36.001397 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.001411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.002226 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.002232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.003796 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.003811 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.004303 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.004311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.004580 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.005315 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:36.005339 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:36.007143 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:36.009240 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:36.009247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:36.009263 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:36.009541 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:36.009871 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:36.009874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:36.009877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:36.010030 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:36.010474 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:36.010483 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:36.010750 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 04:45:36.010958 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 04:45:36.011315 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":144,"threads_started":1}
I20260808 04:45:36.013743 28961 heartbeater.cc:347] Connected to a master server at 127.22.2.190:39375
I20260808 04:45:36.014014 28961 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:36.014554 28961 heartbeater.cc:511] Master 127.22.2.190:39375 requested a full tablet report, sending...
I20260808 04:45:36.016278 28686 ts_manager.cc:194] Registered new tserver with Master: db98b8b20ddc40838de0e585161909b0 (127.22.2.129:40621)
I20260808 04:45:36.017553 28686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:35139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:36.027506 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39375
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:36.144107 28966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:36.144404 28966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:36.144479 28966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:36.148622 28966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:36.148697 28966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:36.148792 28966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:36.153792 28966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:36.154794 28966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:36.155792 28966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:36.165126 28976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:36.165278 28973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:36.165249 28966 server_base.cc:1034] running on GCE node
W20260808 04:45:36.165126 28979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:36.165813 28966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:36.166392 28966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:36.167572 28966 hybrid_clock.cc:648] HybridClock initialized: now 1786164336167557 us; error 25 us; skew 500 ppm
Aug 08 04:45:36 dist-test-slave-04bb krb5kdc[28621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164336, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:36.170292 28966 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:36.171011 28966 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:36.171101 28966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:36.171319 28966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:36.173369 28966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "005ce9fa3f6a4520ddef885b050de448"
server_key_iv: "e67d9a1438cdd7a45465a1dbb9351ee1"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.173914 28966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "005ce9fa3f6a4520ddef885b050de448"
server_key_iv: "e67d9a1438cdd7a45465a1dbb9351ee1"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.177644 28966 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:45:36.180090 28988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:36.181412 28966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:36.181550 28966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "005ce9fa3f6a4520ddef885b050de448"
server_key_iv: "e67d9a1438cdd7a45465a1dbb9351ee1"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.181656 28966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:36.200424 28966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:36.203510 28966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:36.203701 28966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:36.204285 28966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:36.205339 28966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:36.205415 28966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:36.205489 28966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:36.205530 28966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:36.214548 28966 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:36005
I20260808 04:45:36.214566 29149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:36005 every 8 connection(s)
I20260808 04:45:36.215250 28966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:36.224342 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 28966
I20260808 04:45:36.224509 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:36.224861 22538 external_mini_cluster.cc:1428] Setting key 2a76c3d015406f0af7c5a2712f27ce62
Aug 08 04:45:36 dist-test-slave-04bb krb5kdc[28621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164336, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:36.229648 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.217134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:47883 (local address 127.22.2.190:39375)
0808 04:45:36.217305 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.217310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.218516 (+ 1206us) server_negotiation.cc:407] Connection header received
0808 04:45:36.219436 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.219440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.219503 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.219606 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:36.220847 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.221392 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.222557 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.222784 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.222906 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.225670 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:36.225674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:36.225676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:36.225703 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:36.227434 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:36.227952 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:36.227958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:36.227962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:36.228059 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:36.228381 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:36.228388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:36.228392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:36.228672 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:36.228788 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.229426 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.229531 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 04:45:36.230854 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.217515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:39375 (local address 127.22.2.130:47883)
0808 04:45:36.218352 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:36.218397 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:36.219209 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:36.219784 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:36.219791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:36.220154 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:36.220677 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.220692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.221529 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.221537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.222422 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.222432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.222952 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.222956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.223128 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.223722 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:36.223739 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:36.225481 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:36.227553 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:36.227559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:36.227571 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:36.227849 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:36.228158 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:36.228160 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:36.228161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:36.228293 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:36.228794 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:36.228803 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:36.229137 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 04:45:36.229363 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 04:45:36.229745 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
I20260808 04:45:36.232084 29150 heartbeater.cc:347] Connected to a master server at 127.22.2.190:39375
I20260808 04:45:36.232344 29150 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:36.232843 29150 heartbeater.cc:511] Master 127.22.2.190:39375 requested a full tablet report, sending...
I20260808 04:45:36.233946 28686 ts_manager.cc:194] Registered new tserver with Master: 98ec9fd6f2cd4681aa3cc14e1b681d16 (127.22.2.130:36005)
I20260808 04:45:36.234519 28686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:47883
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:36.255071 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39375
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:36.368744 29156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:36.369000 29156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:36.369086 29156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:36.373031 29156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:36.373095 29156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:36.373180 29156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:36.378499 29156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:36.379532 29156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:36.380510 29156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:36.390147 29165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:36.390147 29167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:36.390249 29169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:36.390614 29156 server_base.cc:1034] running on GCE node
I20260808 04:45:36.391022 29156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:36.391618 29156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:36.392773 29156 hybrid_clock.cc:648] HybridClock initialized: now 1786164336392757 us; error 26 us; skew 500 ppm
Aug 08 04:45:36 dist-test-slave-04bb krb5kdc[28621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164336, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:36.396153 29156 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:36.396966 29156 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:36.397060 29156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:36.397271 29156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:36.399192 29156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "16371ee69cf041ebb5aaf1f15328c98b"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "4e9b0c64c37755578f674e3e107dcc03"
server_key_iv: "57825e5027321e5b455067af4a3c04ca"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.399748 29156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "16371ee69cf041ebb5aaf1f15328c98b"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "4e9b0c64c37755578f674e3e107dcc03"
server_key_iv: "57825e5027321e5b455067af4a3c04ca"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.403359 29156 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 04:45:36.405845 29178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:36.407095 29156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:36.407264 29156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "16371ee69cf041ebb5aaf1f15328c98b"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "4e9b0c64c37755578f674e3e107dcc03"
server_key_iv: "57825e5027321e5b455067af4a3c04ca"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.407377 29156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:36.418394 29156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:36.421746 29156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:36.421952 29156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:36.422538 29156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:36.423419 29156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:36.423494 29156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:36.423569 29156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:36.423620 29156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:36.433166 29156 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:39539
I20260808 04:45:36.433183 29352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:39539 every 8 connection(s)
I20260808 04:45:36.433756 29156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:36.441447 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 29156
I20260808 04:45:36.441589 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:36.441972 22538 external_mini_cluster.cc:1428] Setting key 64b1264ee95d7f7da54d64143a57e629
Aug 08 04:45:36 dist-test-slave-04bb krb5kdc[28621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164336, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:36.447489 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.435475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35177 (local address 127.22.2.190:39375)
0808 04:45:36.435634 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.435638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.436464 (+ 826us) server_negotiation.cc:407] Connection header received
0808 04:45:36.437751 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.437754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.437801 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.437881 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:36.438969 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.439474 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.440565 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.440787 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.440881 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.443786 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:36.443789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:36.443791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:36.443812 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:36.445454 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:36.445965 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:36.445968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:36.445970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:36.446058 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:36.446389 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:36.446394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:36.446396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:36.446623 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:36.446732 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.447195 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.447324 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 04:45:36.448210 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.435825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:39375 (local address 127.22.2.131:35177)
0808 04:45:36.436313 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:36.436353 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:36.437556 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:36.438033 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:36.438039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:36.438406 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:36.438849 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.438862 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.439576 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.439578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.440421 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.440429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.440925 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.440928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.441065 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.441766 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:36.441790 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:36.443641 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:36.445576 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:36.445582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:36.445596 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:36.445855 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:36.446147 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:36.446151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:36.446153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:36.446292 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:36.446731 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:36.446737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:36.446971 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 04:45:36.447151 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 04:45:36.447446 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":117,"threads_started":1}
I20260808 04:45:36.449200 29354 heartbeater.cc:347] Connected to a master server at 127.22.2.190:39375
I20260808 04:45:36.449430 29354 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:36.449867 29354 heartbeater.cc:511] Master 127.22.2.190:39375 requested a full tablet report, sending...
I20260808 04:45:36.450843 28686 ts_manager.cc:194] Registered new tserver with Master: 16371ee69cf041ebb5aaf1f15328c98b (127.22.2.131:39539)
I20260808 04:45:36.451402 28686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:35177
I20260808 04:45:36.455647 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:45:36.465374 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.457581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.22.2.190:39375)
0808 04:45:36.457779 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.457785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.457943 (+ 158us) server_negotiation.cc:407] Connection header received
0808 04:45:36.458146 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.458149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.458188 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.458305 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:36.459063 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.459562 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.460430 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.460712 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.460831 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.462268 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:36.462272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:36.462275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:36.462302 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:36.463538 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:36.464002 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:36.464006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:36.464008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:36.464098 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:36.464394 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:36.464399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:36.464401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:36.464712 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:36.464863 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.465102 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.465216 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 04:45:36.468124 28686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:36.471117 28686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:36.485154 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.479291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45848 (local address 127.22.2.129:40621)
0808 04:45:36.479907 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.479914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.480286 (+ 372us) server_negotiation.cc:407] Connection header received
0808 04:45:36.480465 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.480472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.480734 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.480904 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:36.481202 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.482119 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.483744 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.484378 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.484497 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.484690 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.484901 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.484995 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":85,"threads_started":1}
I20260808 04:45:36.485152 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.479181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:40621 (local address 127.0.0.1:45848)
0808 04:45:36.480089 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:36.480137 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:36.480275 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:36.480903 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:36.480909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:36.480934 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:36.481073 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.481083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.482235 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.482240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.483616 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.483626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.484509 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.484513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.484666 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.484710 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 04:45:36.484845 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 04:45:36.485007 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":61,"thread_start_us":78,"threads_started":1}
I20260808 04:45:36.485598 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.480005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45684 (local address 127.22.2.131:39539)
0808 04:45:36.480472 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.480483 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.480671 (+ 188us) server_negotiation.cc:407] Connection header received
0808 04:45:36.480778 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.480783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.480989 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.481150 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:36.481394 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.482498 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.483955 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.484773 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.484907 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.485034 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.485347 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.485447 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":219,"threads_started":1}
I20260808 04:45:36.486039 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.479252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:39539 (local address 127.0.0.1:45684)
0808 04:45:36.480565 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:36.480579 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:36.480670 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:36.481134 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:36.481136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:36.481166 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:36.481268 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.481274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.482654 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.482657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.483831 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.483842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.485096 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.485100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.485189 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.485212 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 04:45:36.485829 (+ 617us) client_negotiation.cc:240] Negotiation successful
0808 04:45:36.485902 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1234,"mutex_wait_us":70,"thread_start_us":90,"threads_started":1}
I20260808 04:45:36.486481 29379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.479593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:36005 (local address 127.0.0.1:52360)
0808 04:45:36.480874 (+ 1281us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:36.480887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:36.480974 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:36.481887 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:36.481890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:36.481910 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:36.481995 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.482000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.483360 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.483364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.484564 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.484576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.486004 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.486007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.486169 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.486202 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 04:45:36.486273 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 04:45:36.486356 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"mutex_wait_us":32,"spinlock_wait_cycles":1408,"thread_start_us":99,"threads_started":1}
I20260808 04:45:36.486917 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.480486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.22.2.130:36005)
0808 04:45:36.481443 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.481454 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.481477 (+ 23us) server_negotiation.cc:407] Connection header received
0808 04:45:36.481549 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.481554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.481740 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.481889 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:36.482089 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.483218 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.484713 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.485695 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.485810 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.485943 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.486648 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.486748 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":175,"threads_started":1}
I20260808 04:45:36.488617 29255 tablet_service.cc:1467] Processing CreateTablet for tablet 8a91462047b34d5087147809f359166d (DEFAULT_TABLE table=test-table [id=fe7939ec9a364f218e9de7229cbadd3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:36.488586 28863 tablet_service.cc:1467] Processing CreateTablet for tablet 8a91462047b34d5087147809f359166d (DEFAULT_TABLE table=test-table [id=fe7939ec9a364f218e9de7229cbadd3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:36.489120 29055 tablet_service.cc:1467] Processing CreateTablet for tablet 8a91462047b34d5087147809f359166d (DEFAULT_TABLE table=test-table [id=fe7939ec9a364f218e9de7229cbadd3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:36.489739 28863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a91462047b34d5087147809f359166d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:36.489756 29255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a91462047b34d5087147809f359166d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:36.490087 29055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a91462047b34d5087147809f359166d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:36.496069 29384 tablet_bootstrap.cc:492] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Bootstrap starting.
I20260808 04:45:36.497829 29386 tablet_bootstrap.cc:492] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0: Bootstrap starting.
I20260808 04:45:36.498499 29384 tablet_bootstrap.cc:654] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:36.499470 29384 log.cc:824] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:36.499532 29386 tablet_bootstrap.cc:654] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:36.500332 29385 tablet_bootstrap.cc:492] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Bootstrap starting.
I20260808 04:45:36.500721 29386 log.cc:824] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:36.501621 29384 tablet_bootstrap.cc:492] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: No bootstrap required, opened a new log
I20260808 04:45:36.501814 29384 ts_tablet_manager.cc:1397] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:45:36.502365 29385 tablet_bootstrap.cc:654] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:36.502718 29386 tablet_bootstrap.cc:492] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0: No bootstrap required, opened a new log
I20260808 04:45:36.502900 29386 ts_tablet_manager.cc:1397] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:45:36.503437 29385 log.cc:824] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:36.505077 29385 tablet_bootstrap.cc:492] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: No bootstrap required, opened a new log
I20260808 04:45:36.505266 29385 ts_tablet_manager.cc:1397] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:45:36.505707 29384 raft_consensus.cc:348] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.505986 29384 raft_consensus.cc:374] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:36.506107 29384 raft_consensus.cc:729] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ec9fd6f2cd4681aa3cc14e1b681d16, State: Initialized, Role: FOLLOWER
I20260808 04:45:36.506582 29384 consensus_queue.cc:260] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.506722 29386 raft_consensus.cc:348] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.507010 29386 raft_consensus.cc:374] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:36.507107 29386 raft_consensus.cc:729] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db98b8b20ddc40838de0e585161909b0, State: Initialized, Role: FOLLOWER
I20260808 04:45:36.507618 29386 consensus_queue.cc:260] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.507717 29384 ts_tablet_manager.cc:1428] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:45:36.508104 29150 heartbeater.cc:503] Master 127.22.2.190:39375 was elected leader, sending a full tablet report...
I20260808 04:45:36.508302 28961 heartbeater.cc:503] Master 127.22.2.190:39375 was elected leader, sending a full tablet report...
I20260808 04:45:36.509490 29386 ts_tablet_manager.cc:1428] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260808 04:45:36.509451 29385 raft_consensus.cc:348] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.509711 29385 raft_consensus.cc:374] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:36.509780 29385 raft_consensus.cc:729] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16371ee69cf041ebb5aaf1f15328c98b, State: Initialized, Role: FOLLOWER
I20260808 04:45:36.510190 29385 consensus_queue.cc:260] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.510839 29354 heartbeater.cc:503] Master 127.22.2.190:39375 was elected leader, sending a full tablet report...
I20260808 04:45:36.511150 29385 ts_tablet_manager.cc:1428] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:45:36.673241 29391 raft_consensus.cc:482] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:36.673441 29391 raft_consensus.cc:504] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.674623 29391 leader_election.cc:288] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db98b8b20ddc40838de0e585161909b0 (127.22.2.129:40621), 16371ee69cf041ebb5aaf1f15328c98b (127.22.2.131:39539)
I20260808 04:45:36.679078 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.675083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:35541 (local address 127.22.2.131:39539)
0808 04:45:36.675231 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.675238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.675265 (+ 27us) server_negotiation.cc:407] Connection header received
0808 04:45:36.675473 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.675477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.675546 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.675630 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:36.676027 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.676654 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.677967 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.678410 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.678507 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.678565 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.678857 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.678944 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 04:45:36.679119 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.675197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53557 (local address 127.22.2.129:40621)
0808 04:45:36.675362 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.675369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.675638 (+ 269us) server_negotiation.cc:407] Connection header received
0808 04:45:36.675711 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.675714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.675773 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.675875 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:36.676222 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.676930 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.678036 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.678552 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.678687 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.678741 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.678913 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.678995 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 04:45:36.679116 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.674932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:39539 (local address 127.22.2.130:35541)
0808 04:45:36.675170 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:36.675191 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:36.675338 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:36.675666 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:36.675670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:36.675694 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:36.675899 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.675908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.676771 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.676775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.677827 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.677838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.678564 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.678569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.678710 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.678734 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 04:45:36.678825 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 04:45:36.678952 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":34}
I20260808 04:45:36.679131 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.675022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:40621 (local address 127.22.2.130:53557)
0808 04:45:36.675505 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:36.675520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:36.675631 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:36.675901 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:36.675904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:36.675925 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:36.676061 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.676068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.677041 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.677044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.677879 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:36.677906 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.678670 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.678673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:36.678753 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.678772 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 04:45:36.678875 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 04:45:36.678981 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":100,"thread_start_us":142,"threads_started":1}
I20260808 04:45:36.679890 28891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a91462047b34d5087147809f359166d" candidate_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db98b8b20ddc40838de0e585161909b0" is_pre_election: true
I20260808 04:45:36.679893 29289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a91462047b34d5087147809f359166d" candidate_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16371ee69cf041ebb5aaf1f15328c98b" is_pre_election: true
I20260808 04:45:36.680214 28891 raft_consensus.cc:2454] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ec9fd6f2cd4681aa3cc14e1b681d16 in term 0.
I20260808 04:45:36.680214 29289 raft_consensus.cc:2454] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ec9fd6f2cd4681aa3cc14e1b681d16 in term 0.
I20260808 04:45:36.680752 28991 leader_election.cc:302] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ec9fd6f2cd4681aa3cc14e1b681d16, db98b8b20ddc40838de0e585161909b0; no voters:
I20260808 04:45:36.681054 29391 raft_consensus.cc:2781] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:36.681139 29391 raft_consensus.cc:482] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:36.681170 29391 raft_consensus.cc:3037] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:36.682260 29391 raft_consensus.cc:504] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.682626 29391 leader_election.cc:288] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [CANDIDATE]: Term 1 election: Requested vote from peers db98b8b20ddc40838de0e585161909b0 (127.22.2.129:40621), 16371ee69cf041ebb5aaf1f15328c98b (127.22.2.131:39539)
I20260808 04:45:36.683007 28891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a91462047b34d5087147809f359166d" candidate_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db98b8b20ddc40838de0e585161909b0"
I20260808 04:45:36.683025 29289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a91462047b34d5087147809f359166d" candidate_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16371ee69cf041ebb5aaf1f15328c98b"
I20260808 04:45:36.683135 28891 raft_consensus.cc:3037] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:36.683174 29289 raft_consensus.cc:3037] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:36.684137 28891 raft_consensus.cc:2454] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ec9fd6f2cd4681aa3cc14e1b681d16 in term 1.
I20260808 04:45:36.684393 29289 raft_consensus.cc:2454] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ec9fd6f2cd4681aa3cc14e1b681d16 in term 1.
I20260808 04:45:36.684489 28991 leader_election.cc:302] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ec9fd6f2cd4681aa3cc14e1b681d16, db98b8b20ddc40838de0e585161909b0; no voters:
W20260808 04:45:36.684716 29355 tablet.cc:2369] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:36.685259 29391 raft_consensus.cc:2781] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:36.685482 29391 raft_consensus.cc:686] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 1 LEADER]: Becoming Leader. State: Replica: 98ec9fd6f2cd4681aa3cc14e1b681d16, State: Running, Role: LEADER
I20260808 04:45:36.685801 29391 consensus_queue.cc:237] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } }
I20260808 04:45:36.689520 28683 catalog_manager.cc:5495] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98ec9fd6f2cd4681aa3cc14e1b681d16 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ec9fd6f2cd4681aa3cc14e1b681d16" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36005 } health_report { overall_health: HEALTHY } } }
W20260808 04:45:36.716338 29151 tablet.cc:2369] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:36.717993 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.714186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.22.2.130:36005)
0808 04:45:36.714329 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.714334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.714349 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:36.714513 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:36.714516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:36.714572 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:36.714681 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:36.714936 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.715433 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.716377 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:36.716704 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:36.716802 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:36.717166 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:36.717569 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:36.717693 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:36.717774 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 04:45:36.717839 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:45:36.724295 28891 raft_consensus.cc:1261] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 1 FOLLOWER]: Refusing update from remote peer 98ec9fd6f2cd4681aa3cc14e1b681d16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:36.725121 29391 consensus_queue.cc:1035] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db98b8b20ddc40838de0e585161909b0" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:36.725239 29289 raft_consensus.cc:1261] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 1 FOLLOWER]: Refusing update from remote peer 98ec9fd6f2cd4681aa3cc14e1b681d16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:36.725852 29391 consensus_queue.cc:1035] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16371ee69cf041ebb5aaf1f15328c98b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:36.732182 29402 mvcc.cc:205] Tried to move back new op lower bound from 7316129123213651968 to 7316129123067486208. Current Snapshot: MvccSnapshot[applied={T|T < 7316129123213651968}]
I20260808 04:45:36.735598 29405 mvcc.cc:205] Tried to move back new op lower bound from 7316129123213651968 to 7316129123067486208. Current Snapshot: MvccSnapshot[applied={T|T < 7316129123213651968}]
I20260808 04:45:36.737833 29403 mvcc.cc:205] Tried to move back new op lower bound from 7316129123213651968 to 7316129123067486208. Current Snapshot: MvccSnapshot[applied={T|T < 7316129123213651968}]
I20260808 04:45:36.741899 28683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:36.742038 28683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:36.744374 28683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 833a2fc56f1f4fbe95ca422fc44258ad: Sending DeleteTablet for 3 replicas of tablet 8a91462047b34d5087147809f359166d
I20260808 04:45:36.745162 28863 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a91462047b34d5087147809f359166d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:36 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:45848
I20260808 04:45:36.745296 29255 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a91462047b34d5087147809f359166d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:36 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:45684
I20260808 04:45:36.745294 29055 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a91462047b34d5087147809f359166d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:36 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:52360
I20260808 04:45:36.745751 29416 tablet_replica.cc:330] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0: stopping tablet replica
I20260808 04:45:36.746022 29418 tablet_replica.cc:330] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: stopping tablet replica
I20260808 04:45:36.746068 29416 raft_consensus.cc:2229] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:36.746403 29416 raft_consensus.cc:2258] T 8a91462047b34d5087147809f359166d P db98b8b20ddc40838de0e585161909b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:36.746387 29418 raft_consensus.cc:2229] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:36.746565 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 28770
I20260808 04:45:36.754578 29418 raft_consensus.cc:2258] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:36.756227 29418 ts_tablet_manager.cc:1905] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:36.759871 29418 ts_tablet_manager.cc:1918] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:36.759981 29418 log.cc:1196] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/8a91462047b34d5087147809f359166d
I20260808 04:45:36.760284 29417 tablet_replica.cc:330] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: stopping tablet replica
I20260808 04:45:36.760361 29418 ts_tablet_manager.cc:1939] T 8a91462047b34d5087147809f359166d P 98ec9fd6f2cd4681aa3cc14e1b681d16: Deleting consensus metadata
I20260808 04:45:36.760708 29417 raft_consensus.cc:2229] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:36.761085 29417 raft_consensus.cc:2258] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:36.761799 28659 catalog_manager.cc:4841] TS 98ec9fd6f2cd4681aa3cc14e1b681d16 (127.22.2.130:36005): tablet 8a91462047b34d5087147809f359166d (table test-table [id=fe7939ec9a364f218e9de7229cbadd3c]) successfully deleted
W20260808 04:45:36.762176 28663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:40621 (error 108)
I20260808 04:45:36.762688 29417 ts_tablet_manager.cc:1905] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:36.763237 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.762910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:40621 (local address 127.0.0.1:45856)
0808 04:45:36.763043 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:36.763123 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:40621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 04:45:36.763378 28663 catalog_manager.cc:4584] TS db98b8b20ddc40838de0e585161909b0 (127.22.2.129:40621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a91462047b34d5087147809f359166d: Network error: Client connection negotiation failed: client connection to 127.22.2.129:40621: connect: Connection refused (error 111)
I20260808 04:45:36.763720 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 28966
I20260808 04:45:36.769994 29417 ts_tablet_manager.cc:1918] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:36.770108 29417 log.cc:1196] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/8a91462047b34d5087147809f359166d
I20260808 04:45:36.770500 29417 ts_tablet_manager.cc:1939] T 8a91462047b34d5087147809f359166d P 16371ee69cf041ebb5aaf1f15328c98b: Deleting consensus metadata
I20260808 04:45:36.771975 28664 catalog_manager.cc:4841] TS 16371ee69cf041ebb5aaf1f15328c98b (127.22.2.131:39539): tablet 8a91462047b34d5087147809f359166d (table test-table [id=fe7939ec9a364f218e9de7229cbadd3c]) successfully deleted
I20260808 04:45:36.782478 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 29156
I20260808 04:45:36.793593 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 28638
2026-08-08T04:45:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2606 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T04:45:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:36Z Disabled control of system clock
I20260808 04:45:36.829972 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36067
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:39209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:36067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 04:45:36.943706 29424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:36.943964 29424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:36.944001 29424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:36.948076 29424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:36.948133 29424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:36.948149 29424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:36.948166 29424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:36.948182 29424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:36.952836 29424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:36067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:36.953816 29424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:36.955129 29424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:36.963551 29433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:36.963673 29435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:36.963840 29437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:36.964018 29424 server_base.cc:1034] running on GCE node
I20260808 04:45:36.964630 29424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:36.965740 29424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:36.966936 29424 hybrid_clock.cc:648] HybridClock initialized: now 1786164336966916 us; error 34 us; skew 500 ppm
I20260808 04:45:36.968991 29424 webserver.cc:460] Webserver started at http://127.22.2.190:38487/ using document root <none> and password file <none>
I20260808 04:45:36.969591 29424 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:36.969684 29424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:36.969920 29424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:36.971982 29424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "c5ef95b3638e23b4f63df67e8db0e57a"
server_key_iv: "3ea767e616f2ea995e503fa109d1e124"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.972604 29424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "c5ef95b3638e23b4f63df67e8db0e57a"
server_key_iv: "3ea767e616f2ea995e503fa109d1e124"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.976289 29424 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:36.978816 29446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:36.980144 29424 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:36.980283 29424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f"
format_stamp: "Formatted at 2026-08-08 04:45:36 on dist-test-slave-04bb"
server_key: "c5ef95b3638e23b4f63df67e8db0e57a"
server_key_iv: "3ea767e616f2ea995e503fa109d1e124"
server_key_version: "encryptionkey@0"
I20260808 04:45:36.980399 29424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:36.985260 29424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:36.985944 29424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:36.986179 29424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:36.994233 29424 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:36067
I20260808 04:45:36.994304 29533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:36067 every 8 connection(s)
I20260808 04:45:36.995278 29424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:36.995900 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 29424
I20260808 04:45:36.996021 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:36.996311 22538 external_mini_cluster.cc:1428] Setting key efc5bf9949a4099edc17dc54a79acf50
I20260808 04:45:36.999176 29537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:37.003295 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:36.997910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.22.2.190:36067)
0808 04:45:36.999249 (+ 1339us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:36.999272 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:36.999332 (+ 60us) server_negotiation.cc:407] Connection header received
0808 04:45:36.999993 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.000009 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.000414 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.000831 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.001739 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.001744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.001757 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.001768 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.001826 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.001842 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.001994 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.002069 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.002347 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":954,"thread_start_us":812,"threads_started":1}
I20260808 04:45:37.006282 29537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f: Bootstrap starting.
I20260808 04:45:37.008829 29537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:37.009832 29537 log.cc:824] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:37.012058 29537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f: No bootstrap required, opened a new log
I20260808 04:45:37.015863 29537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36067 } }
I20260808 04:45:37.016089 29537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:37.016178 29537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf2288ec2dc4e6d9dd22cc6e3f0269f, State: Initialized, Role: FOLLOWER
I20260808 04:45:37.016685 29537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36067 } }
I20260808 04:45:37.016809 29537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:37.016861 29537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:37.016930 29537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:37.031414 29537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36067 } }
I20260808 04:45:37.031904 29537 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bf2288ec2dc4e6d9dd22cc6e3f0269f; no voters:
I20260808 04:45:37.032284 29537 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:37.032425 29546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:37.032713 29546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [term 1 LEADER]: Becoming Leader. State: Replica: 0bf2288ec2dc4e6d9dd22cc6e3f0269f, State: Running, Role: LEADER
I20260808 04:45:37.033092 29546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36067 } }
I20260808 04:45:37.033913 29537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:37.035202 29547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36067 } } }
I20260808 04:45:37.035348 29547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:37.035771 29548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf2288ec2dc4e6d9dd22cc6e3f0269f. Latest consensus state: current_term: 1 leader_uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf2288ec2dc4e6d9dd22cc6e3f0269f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36067 } } }
I20260808 04:45:37.035967 29548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f [sys.catalog]: This master's current role is: LEADER
W20260808 04:45:37.039054 29566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:45:37.039124 29566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:45:37.039219 29559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:37.040148 29559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:37.046183 29559 catalog_manager.cc:1347] Generated new cluster ID: c918909ed45e48c08fbced2081eda40e
I20260808 04:45:37.046260 29559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:37.063805 29559 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:37.064127 29559 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:37.070657 29559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f: Generated new TSK 0
I20260808 04:45:37.071303 29559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:45:37.080312 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36067
--builtin_ntp_servers=127.22.2.148:39209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 04:45:37.206516 29572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:37.206755 29572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:37.206794 29572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:37.210902 29572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:37.210954 29572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:37.211040 29572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:37.215755 29572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:37.216912 29572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:37.218391 29572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:37.229332 29582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:37.229332 29578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:37.229372 29579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:37.229760 29572 server_base.cc:1034] running on GCE node
I20260808 04:45:37.230142 29572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:37.230741 29572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:37.231910 29572 hybrid_clock.cc:648] HybridClock initialized: now 1786164337231885 us; error 37 us; skew 500 ppm
I20260808 04:45:37.234053 29572 webserver.cc:460] Webserver started at http://127.22.2.129:34355/ using document root <none> and password file <none>
I20260808 04:45:37.234732 29572 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:37.234851 29572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:37.235105 29572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:37.237393 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "f676f4a9007c420da68996abbb8f8e84"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "47ec473b2c9f2376524ad21c49bc7a54"
server_key_iv: "bc2845becc087d1846b8a97051a2e67e"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.237979 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "f676f4a9007c420da68996abbb8f8e84"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "47ec473b2c9f2376524ad21c49bc7a54"
server_key_iv: "bc2845becc087d1846b8a97051a2e67e"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.241931 29572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:45:37.244634 29588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.245882 29572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:37.246013 29572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "f676f4a9007c420da68996abbb8f8e84"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "47ec473b2c9f2376524ad21c49bc7a54"
server_key_iv: "bc2845becc087d1846b8a97051a2e67e"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.246130 29572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:37.250918 29572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:37.251516 29572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:37.251694 29572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:37.252266 29572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:37.253248 29572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:37.253324 29572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.253388 29572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:37.253432 29572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.262862 29572 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:35257
I20260808 04:45:37.262928 29751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:35257 every 8 connection(s)
I20260808 04:45:37.263921 29572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:37.268491 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 29572
I20260808 04:45:37.268662 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:37.269084 22538 external_mini_cluster.cc:1428] Setting key 6dc66d1106b5095c7860f8366396507e
I20260808 04:45:37.271299 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36067
--builtin_ntp_servers=127.22.2.148:39209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 04:45:37.271389 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.266278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:44123 (local address 127.22.2.190:36067)
0808 04:45:37.266442 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.266447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.267538 (+ 1091us) server_negotiation.cc:407] Connection header received
0808 04:45:37.269411 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.269418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.269470 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.269572 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.270865 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.270868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.270870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.270908 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.270928 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.270936 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.271018 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.271216 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.271253 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 04:45:37.272617 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.266646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36067 (local address 127.22.2.129:44123)
0808 04:45:37.267382 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.267425 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.269148 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.269775 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.269782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.269874 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.270527 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.270532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.270591 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.270684 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.271039 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.271053 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.271061 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.271269 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.271537 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":147,"threads_started":1}
I20260808 04:45:37.273983 29752 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36067
I20260808 04:45:37.274269 29752 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:37.274694 29752 heartbeater.cc:511] Master 127.22.2.190:36067 requested a full tablet report, sending...
I20260808 04:45:37.276350 29471 ts_manager.cc:194] Registered new tserver with Master: f676f4a9007c420da68996abbb8f8e84 (127.22.2.129:35257)
W20260808 04:45:37.392731 29757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:37.392961 29757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:37.393025 29757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:37.396978 29757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:37.397029 29757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:37.397128 29757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:37.401723 29757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:37.402771 29757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:37.404152 29757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:37.413707 29767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:37.413645 29762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:37.413659 29763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:37.413906 29757 server_base.cc:1034] running on GCE node
I20260808 04:45:37.414613 29757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:37.415319 29757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:37.416580 29757 hybrid_clock.cc:648] HybridClock initialized: now 1786164337416547 us; error 39 us; skew 500 ppm
I20260808 04:45:37.418713 29757 webserver.cc:460] Webserver started at http://127.22.2.130:36277/ using document root <none> and password file <none>
I20260808 04:45:37.419381 29757 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:37.419481 29757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:37.419756 29757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:37.422127 29757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "e38c32386c3a4a9c9006ce7e3a25bb34"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "8379681eece6ed1f5d8ebd5fe579192f"
server_key_iv: "8710c8a8f7faba4fcf1eb521805ce429"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.422758 29757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "e38c32386c3a4a9c9006ce7e3a25bb34"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "8379681eece6ed1f5d8ebd5fe579192f"
server_key_iv: "8710c8a8f7faba4fcf1eb521805ce429"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.427392 29757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 04:45:37.430939 29777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.432335 29757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:37.432482 29757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "e38c32386c3a4a9c9006ce7e3a25bb34"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "8379681eece6ed1f5d8ebd5fe579192f"
server_key_iv: "8710c8a8f7faba4fcf1eb521805ce429"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.432655 29757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:37.438877 29757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:37.439571 29757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:37.439771 29757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:37.440390 29757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:37.441458 29757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:37.441532 29757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.441604 29757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:37.441653 29757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.451655 29757 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:43509
I20260808 04:45:37.451686 29940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:43509 every 8 connection(s)
I20260808 04:45:37.452643 29757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:37.459180 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.454675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:58321 (local address 127.22.2.190:36067)
0808 04:45:37.454818 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.454822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.455942 (+ 1120us) server_negotiation.cc:407] Connection header received
0808 04:45:37.457373 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.457377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.457423 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.457527 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.458656 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.458659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.458662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.458675 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.458696 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.458711 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.458808 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.458952 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.459001 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:37.459371 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 29757
I20260808 04:45:37.459523 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:37.459816 22538 external_mini_cluster.cc:1428] Setting key a9534234c6ccc73577a49775cf533305
I20260808 04:45:37.460196 29944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.455096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36067 (local address 127.22.2.130:58321)
0808 04:45:37.455799 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.455839 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.457158 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.457691 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.457698 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.457783 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.458390 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.458392 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.458432 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.458499 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.458792 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.458801 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.458807 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.458983 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.459219 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":116,"threads_started":1}
I20260808 04:45:37.461380 29941 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36067
I20260808 04:45:37.461629 29941 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:37.462101 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36067
--builtin_ntp_servers=127.22.2.148:39209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 04:45:37.462133 29941 heartbeater.cc:511] Master 127.22.2.190:36067 requested a full tablet report, sending...
I20260808 04:45:37.463277 29471 ts_manager.cc:194] Registered new tserver with Master: e38c32386c3a4a9c9006ce7e3a25bb34 (127.22.2.130:43509)
W20260808 04:45:37.582366 29947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:37.582644 29947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:37.582719 29947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:37.586818 29947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:37.586897 29947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:37.586997 29947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:37.591737 29947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:37.592856 29947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:37.594282 29947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:37.604423 29953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:37.604449 29957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:37.604423 29952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:37.605536 29947 server_base.cc:1034] running on GCE node
I20260808 04:45:37.605942 29947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:37.606619 29947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:37.607810 29947 hybrid_clock.cc:648] HybridClock initialized: now 1786164337607792 us; error 31 us; skew 500 ppm
I20260808 04:45:37.609812 29947 webserver.cc:460] Webserver started at http://127.22.2.131:37811/ using document root <none> and password file <none>
I20260808 04:45:37.610392 29947 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:37.610482 29947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:37.610703 29947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:37.612830 29947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "fb69b300db304e049397e24ff1019a4a"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "f93e12ad337e256411a80a5d16ae88a0"
server_key_iv: "b388d97f113c8fbd36c6c0e718cbccb8"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.613406 29947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "fb69b300db304e049397e24ff1019a4a"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "f93e12ad337e256411a80a5d16ae88a0"
server_key_iv: "b388d97f113c8fbd36c6c0e718cbccb8"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.617167 29947 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:45:37.619921 29965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.621348 29947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:37.621487 29947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "fb69b300db304e049397e24ff1019a4a"
format_stamp: "Formatted at 2026-08-08 04:45:37 on dist-test-slave-04bb"
server_key: "f93e12ad337e256411a80a5d16ae88a0"
server_key_iv: "b388d97f113c8fbd36c6c0e718cbccb8"
server_key_version: "encryptionkey@0"
I20260808 04:45:37.621604 29947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:37.626348 29947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:37.626952 29947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:37.627135 29947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:37.627665 29947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:37.628669 29947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:37.628731 29947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.628795 29947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:37.628849 29947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:37.639753 29947 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:35381
I20260808 04:45:37.639779 30131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:35381 every 8 connection(s)
I20260808 04:45:37.640744 29947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:37.646765 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.642665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:45229 (local address 127.22.2.190:36067)
0808 04:45:37.642812 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.642816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.643851 (+ 1035us) server_negotiation.cc:407] Connection header received
0808 04:45:37.645193 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.645196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.645236 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.645318 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.646371 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.646374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.646376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.646388 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.646407 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.646415 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.646493 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.646608 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.646655 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:45:37.647629 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.643052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36067 (local address 127.22.2.131:45229)
0808 04:45:37.643687 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.643725 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.644994 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.645479 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.645486 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.645570 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.646102 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.646104 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.646143 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.646228 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.646488 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.646495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.646499 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.646628 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.646812 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":103,"threads_started":1}
I20260808 04:45:37.648874 30132 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36067
I20260808 04:45:37.649163 30132 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:37.649421 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 29947
I20260808 04:45:37.649569 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:37.649713 30132 heartbeater.cc:511] Master 127.22.2.190:36067 requested a full tablet report, sending...
I20260808 04:45:37.649926 22538 external_mini_cluster.cc:1428] Setting key d314388719540f4e3b8220773c84a28a
I20260808 04:45:37.650919 29467 ts_manager.cc:194] Registered new tserver with Master: fb69b300db304e049397e24ff1019a4a (127.22.2.131:35381)
I20260808 04:45:37.652264 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:45:37.655977 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.654323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.22.2.190:36067)
0808 04:45:37.654449 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.654453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.654672 (+ 219us) server_negotiation.cc:407] Connection header received
0808 04:45:37.654805 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.654808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.654841 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.654916 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.655670 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.655673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.655691 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.655702 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.655719 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.655727 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.655809 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.655854 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.655885 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 04:45:37.658983 29467 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:37.662089 29467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:37.674453 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.670815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:43509 (local address 127.0.0.1:41870)
0808 04:45:37.672014 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.672025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.672134 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.673238 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.673241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.673258 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.673666 (+ 408us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.673668 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.673679 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.673706 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.674110 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.674113 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.674117 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.674202 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.674293 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"thread_start_us":108,"threads_started":1}
I20260808 04:45:37.674467 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.671455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.22.2.129:35257)
0808 04:45:37.671864 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.671881 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.672173 (+ 292us) server_negotiation.cc:407] Connection header received
0808 04:45:37.672359 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.672365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.672505 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.672655 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.673775 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.673782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.673796 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.673810 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.673863 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.673886 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.673952 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.674231 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.674314 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":115,"threads_started":1}
I20260808 04:45:37.674505 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.670620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35381 (local address 127.0.0.1:38554)
0808 04:45:37.671693 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.671772 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.671878 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.672603 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.672609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.672694 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.673450 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.673454 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.673477 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.673555 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.674303 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.674306 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.674309 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.674378 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.674406 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"thread_start_us":97,"threads_started":1}
I20260808 04:45:37.674804 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.671042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35257 (local address 127.0.0.1:51208)
0808 04:45:37.672076 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.672089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.672237 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.672997 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.673002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.673020 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.673486 (+ 466us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.673488 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.673498 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.673586 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.674052 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.674062 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.674067 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.674632 (+ 565us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.674680 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"thread_start_us":113,"threads_started":1}
I20260808 04:45:37.674871 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.671455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.22.2.130:43509)
0808 04:45:37.671864 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.671880 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.672129 (+ 249us) server_negotiation.cc:407] Connection header received
0808 04:45:37.672229 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.672235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.672441 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.672605 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.673837 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.673843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.673855 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.673867 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.673915 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.673949 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.674061 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.674628 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.674704 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":111,"threads_started":1}
I20260808 04:45:37.674990 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.670776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38554 (local address 127.22.2.131:35381)
0808 04:45:37.672132 (+ 1356us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.672146 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.672167 (+ 21us) server_negotiation.cc:407] Connection header received
0808 04:45:37.672234 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.672246 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.672432 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.672604 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.674049 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.674054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.674063 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.674076 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.674115 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.674139 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.674261 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.674776 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.674837 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1238,"thread_start_us":114,"threads_started":1}
I20260808 04:45:37.677659 29847 tablet_service.cc:1467] Processing CreateTablet for tablet d1ed5cb57e13474b88ec9ce7a0ae5490 (DEFAULT_TABLE table=test-table [id=4ac4418cccc04c398b71b032cdd97ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:37.677659 29656 tablet_service.cc:1467] Processing CreateTablet for tablet d1ed5cb57e13474b88ec9ce7a0ae5490 (DEFAULT_TABLE table=test-table [id=4ac4418cccc04c398b71b032cdd97ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:37.677604 30037 tablet_service.cc:1467] Processing CreateTablet for tablet d1ed5cb57e13474b88ec9ce7a0ae5490 (DEFAULT_TABLE table=test-table [id=4ac4418cccc04c398b71b032cdd97ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:37.678776 30037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ed5cb57e13474b88ec9ce7a0ae5490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:37.678777 29656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ed5cb57e13474b88ec9ce7a0ae5490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:37.678776 29847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ed5cb57e13474b88ec9ce7a0ae5490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:37.685747 30156 tablet_bootstrap.cc:492] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: Bootstrap starting.
I20260808 04:45:37.686421 30157 tablet_bootstrap.cc:492] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: Bootstrap starting.
I20260808 04:45:37.686990 30158 tablet_bootstrap.cc:492] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84: Bootstrap starting.
I20260808 04:45:37.688838 30157 tablet_bootstrap.cc:654] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:37.688943 30158 tablet_bootstrap.cc:654] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:37.689322 30156 tablet_bootstrap.cc:654] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:37.689702 30158 log.cc:824] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:37.689738 30157 log.cc:824] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:37.690089 30156 log.cc:824] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:37.691748 30156 tablet_bootstrap.cc:492] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: No bootstrap required, opened a new log
I20260808 04:45:37.691776 30158 tablet_bootstrap.cc:492] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84: No bootstrap required, opened a new log
I20260808 04:45:37.691776 30157 tablet_bootstrap.cc:492] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: No bootstrap required, opened a new log
I20260808 04:45:37.691958 30157 ts_tablet_manager.cc:1397] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:45:37.691958 30158 ts_tablet_manager.cc:1397] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 04:45:37.691960 30156 ts_tablet_manager.cc:1397] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:45:37.695400 30156 raft_consensus.cc:348] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.695645 30156 raft_consensus.cc:374] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:37.695699 30156 raft_consensus.cc:729] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69b300db304e049397e24ff1019a4a, State: Initialized, Role: FOLLOWER
I20260808 04:45:37.696631 30156 consensus_queue.cc:260] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.696691 30157 raft_consensus.cc:348] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.696691 30158 raft_consensus.cc:348] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.696944 30158 raft_consensus.cc:374] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:37.696944 30157 raft_consensus.cc:374] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:37.697002 30157 raft_consensus.cc:729] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38c32386c3a4a9c9006ce7e3a25bb34, State: Initialized, Role: FOLLOWER
I20260808 04:45:37.697002 30158 raft_consensus.cc:729] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f676f4a9007c420da68996abbb8f8e84, State: Initialized, Role: FOLLOWER
I20260808 04:45:37.697500 30132 heartbeater.cc:503] Master 127.22.2.190:36067 was elected leader, sending a full tablet report...
I20260808 04:45:37.697474 30158 consensus_queue.cc:260] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.697932 30156 ts_tablet_manager.cc:1428] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:45:37.698210 29752 heartbeater.cc:503] Master 127.22.2.190:36067 was elected leader, sending a full tablet report...
I20260808 04:45:37.698347 30157 consensus_queue.cc:260] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.699029 29941 heartbeater.cc:503] Master 127.22.2.190:36067 was elected leader, sending a full tablet report...
I20260808 04:45:37.699637 30158 ts_tablet_manager.cc:1428] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 04:45:37.700961 30157 ts_tablet_manager.cc:1428] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
W20260808 04:45:37.703666 29942 tablet.cc:2369] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:45:37.765239 29753 tablet.cc:2369] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:37.888906 30172 raft_consensus.cc:482] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:37.889267 30172 raft_consensus.cc:504] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.890595 30172 leader_election.cc:288] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69b300db304e049397e24ff1019a4a (127.22.2.131:35381), f676f4a9007c420da68996abbb8f8e84 (127.22.2.129:35257)
W20260808 04:45:37.891800 30133 tablet.cc:2369] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:37.896127 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.891002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:59139 (local address 127.22.2.129:35257)
0808 04:45:37.891136 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.891140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.891157 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:45:37.891270 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.891293 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.891338 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.891426 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.895668 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.895672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.895675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.895691 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.895716 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.895727 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.895842 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.895943 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.895987 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 04:45:37.896127 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.891023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:42661 (local address 127.22.2.131:35381)
0808 04:45:37.891140 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.891146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.891540 (+ 394us) server_negotiation.cc:407] Connection header received
0808 04:45:37.891676 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.891681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.891734 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.891840 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.895668 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.895678 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.895680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.895692 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.895716 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.895727 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.895842 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.895943 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.895987 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 04:45:37.896158 29944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.890907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35257 (local address 127.22.2.130:59139)
0808 04:45:37.891041 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.891059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.891156 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.891501 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.891504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.891535 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.895468 (+ 3933us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.895470 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.895479 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.895512 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.895836 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.895839 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.895843 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.895933 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.895999 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":19}
I20260808 04:45:37.896171 30174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.890915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35381 (local address 127.22.2.130:42661)
0808 04:45:37.891447 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.891461 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.891571 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.891862 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.891865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.891879 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.895464 (+ 3585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.895466 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.895479 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.895512 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.895836 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.895839 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.895843 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.895933 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.895988 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":164,"thread_start_us":157,"threads_started":1}
I20260808 04:45:37.896610 30168 raft_consensus.cc:482] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:37.896813 30168 raft_consensus.cc:504] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.897024 30064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ed5cb57e13474b88ec9ce7a0ae5490" candidate_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69b300db304e049397e24ff1019a4a" is_pre_election: true
I20260808 04:45:37.897076 29695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ed5cb57e13474b88ec9ce7a0ae5490" candidate_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f676f4a9007c420da68996abbb8f8e84" is_pre_election: true
I20260808 04:45:37.897329 29695 raft_consensus.cc:2454] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38c32386c3a4a9c9006ce7e3a25bb34 in term 0.
I20260808 04:45:37.897310 30064 raft_consensus.cc:2454] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38c32386c3a4a9c9006ce7e3a25bb34 in term 0.
I20260808 04:45:37.897866 29779 leader_election.cc:302] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e38c32386c3a4a9c9006ce7e3a25bb34, f676f4a9007c420da68996abbb8f8e84; no voters:
I20260808 04:45:37.898160 30172 raft_consensus.cc:2781] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:37.898344 30172 raft_consensus.cc:482] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:37.898464 30172 raft_consensus.cc:3037] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:37.898577 30168 leader_election.cc:288] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69b300db304e049397e24ff1019a4a (127.22.2.131:35381), e38c32386c3a4a9c9006ce7e3a25bb34 (127.22.2.130:43509)
I20260808 04:45:37.900301 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.898370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:35775 (local address 127.22.2.131:35381)
0808 04:45:37.898490 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.898494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.898564 (+ 70us) server_negotiation.cc:407] Connection header received
0808 04:45:37.898759 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.898764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.898803 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.898855 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.899844 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.899848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.899850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.899861 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.899886 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.899893 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.899991 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.900110 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.900172 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:45:37.900288 30175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.898260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35381 (local address 127.22.2.129:35775)
0808 04:45:37.898480 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.898494 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.898625 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.898982 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.898984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.898997 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.899690 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.899692 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.899705 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.899748 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.900001 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.900007 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.900010 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.900093 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.900131 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":77,"threads_started":1}
I20260808 04:45:37.900650 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.898849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:59929 (local address 127.22.2.130:43509)
0808 04:45:37.898960 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.898965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.899353 (+ 388us) server_negotiation.cc:407] Connection header received
0808 04:45:37.899450 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.899454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.899488 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.899588 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.900211 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.900215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.900217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.900228 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.900246 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.900255 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.900366 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.900460 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.900507 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:45:37.900697 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.898803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:43509 (local address 127.22.2.129:59929)
0808 04:45:37.899255 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:37.899265 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:37.899367 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:37.899605 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:37.899608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:37.899622 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:37.900061 (+ 439us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:45:37.900065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:37.900077 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:45:37.900100 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:37.900370 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:37.900375 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:37.900379 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:45:37.900471 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 04:45:37.900508 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":87,"threads_started":1}
I20260808 04:45:37.900966 30064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ed5cb57e13474b88ec9ce7a0ae5490" candidate_uuid: "f676f4a9007c420da68996abbb8f8e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69b300db304e049397e24ff1019a4a" is_pre_election: true
I20260808 04:45:37.901104 30064 raft_consensus.cc:2454] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f676f4a9007c420da68996abbb8f8e84 in term 0.
I20260808 04:45:37.901183 29874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ed5cb57e13474b88ec9ce7a0ae5490" candidate_uuid: "f676f4a9007c420da68996abbb8f8e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" is_pre_election: true
I20260808 04:45:37.901520 29590 leader_election.cc:302] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f676f4a9007c420da68996abbb8f8e84, fb69b300db304e049397e24ff1019a4a; no voters:
I20260808 04:45:37.901808 30168 raft_consensus.cc:2781] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:37.901880 30168 raft_consensus.cc:482] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:37.901932 30168 raft_consensus.cc:3037] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:37.934432 30168 raft_consensus.cc:504] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.934432 30172 raft_consensus.cc:504] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.934922 29874 raft_consensus.cc:2379] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f676f4a9007c420da68996abbb8f8e84 in current term 1: Already voted for candidate e38c32386c3a4a9c9006ce7e3a25bb34 in this term.
I20260808 04:45:37.935617 30064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ed5cb57e13474b88ec9ce7a0ae5490" candidate_uuid: "f676f4a9007c420da68996abbb8f8e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69b300db304e049397e24ff1019a4a"
I20260808 04:45:37.935721 29695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ed5cb57e13474b88ec9ce7a0ae5490" candidate_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f676f4a9007c420da68996abbb8f8e84"
I20260808 04:45:37.935765 30064 raft_consensus.cc:3037] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:37.935938 29695 raft_consensus.cc:2379] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e38c32386c3a4a9c9006ce7e3a25bb34 in current term 1: Already voted for candidate f676f4a9007c420da68996abbb8f8e84 in this term.
I20260808 04:45:37.936808 30172 leader_election.cc:288] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [CANDIDATE]: Term 1 election: Requested vote from peers fb69b300db304e049397e24ff1019a4a (127.22.2.131:35381), f676f4a9007c420da68996abbb8f8e84 (127.22.2.129:35257)
I20260808 04:45:37.937122 30168 leader_election.cc:288] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [CANDIDATE]: Term 1 election: Requested vote from peers fb69b300db304e049397e24ff1019a4a (127.22.2.131:35381), e38c32386c3a4a9c9006ce7e3a25bb34 (127.22.2.130:43509)
I20260808 04:45:37.937222 30062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ed5cb57e13474b88ec9ce7a0ae5490" candidate_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69b300db304e049397e24ff1019a4a"
I20260808 04:45:37.937320 30064 raft_consensus.cc:2454] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f676f4a9007c420da68996abbb8f8e84 in term 1.
I20260808 04:45:37.937651 29590 leader_election.cc:302] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f676f4a9007c420da68996abbb8f8e84, fb69b300db304e049397e24ff1019a4a; no voters:
I20260808 04:45:37.937650 29874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ed5cb57e13474b88ec9ce7a0ae5490" candidate_uuid: "f676f4a9007c420da68996abbb8f8e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34"
I20260808 04:45:37.937808 29874 raft_consensus.cc:2379] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f676f4a9007c420da68996abbb8f8e84 in current term 1: Already voted for candidate e38c32386c3a4a9c9006ce7e3a25bb34 in this term.
I20260808 04:45:37.937835 30168 raft_consensus.cc:2781] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:37.937841 29778 leader_election.cc:302] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38c32386c3a4a9c9006ce7e3a25bb34; no voters: f676f4a9007c420da68996abbb8f8e84, fb69b300db304e049397e24ff1019a4a
I20260808 04:45:37.938069 30168 raft_consensus.cc:686] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [term 1 LEADER]: Becoming Leader. State: Replica: f676f4a9007c420da68996abbb8f8e84, State: Running, Role: LEADER
I20260808 04:45:37.938138 30172 raft_consensus.cc:2726] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 04:45:37.938467 30168 consensus_queue.cc:237] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } }
I20260808 04:45:37.942169 29467 catalog_manager.cc:5495] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 reported cstate change: term changed from 0 to 1, leader changed from <none> to f676f4a9007c420da68996abbb8f8e84 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "f676f4a9007c420da68996abbb8f8e84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f676f4a9007c420da68996abbb8f8e84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35257 } health_report { overall_health: HEALTHY } } }
I20260808 04:45:37.972285 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:37.970761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.22.2.129:35257)
0808 04:45:37.970895 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:37.970899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:37.970914 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:37.971059 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:37.971062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:37.971143 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:37.971222 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:37.971940 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:37.971943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:37.971945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:37.971956 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:37.971976 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:37.971984 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:37.972058 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:37.972140 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 04:45:37.972176 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:45:37.978591 30062 raft_consensus.cc:1261] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 1 FOLLOWER]: Refusing update from remote peer f676f4a9007c420da68996abbb8f8e84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:37.978586 29874 raft_consensus.cc:1261] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 1 FOLLOWER]: Refusing update from remote peer f676f4a9007c420da68996abbb8f8e84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:37.979235 30180 consensus_queue.cc:1035] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38c32386c3a4a9c9006ce7e3a25bb34" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:37.979476 30168 consensus_queue.cc:1035] T d1ed5cb57e13474b88ec9ce7a0ae5490 P f676f4a9007c420da68996abbb8f8e84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69b300db304e049397e24ff1019a4a" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:37.987898 30185 mvcc.cc:205] Tried to move back new op lower bound from 7316129128352473088 to 7316129128198901760. Current Snapshot: MvccSnapshot[applied={T|T < 7316129128352473088}]
I20260808 04:45:37.998369 29466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:37.998544 29466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:38.001025 29466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bf2288ec2dc4e6d9dd22cc6e3f0269f: Sending DeleteTablet for 3 replicas of tablet d1ed5cb57e13474b88ec9ce7a0ae5490
I20260808 04:45:38.001816 30037 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ed5cb57e13474b88ec9ce7a0ae5490 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:37 UTC) from {username='slave'} at 127.0.0.1:38554
I20260808 04:45:38.001919 29656 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ed5cb57e13474b88ec9ce7a0ae5490 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:37 UTC) from {username='slave'} at 127.0.0.1:51208
I20260808 04:45:38.002504 29847 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ed5cb57e13474b88ec9ce7a0ae5490 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:37 UTC) from {username='slave'} at 127.0.0.1:41870
I20260808 04:45:38.002688 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 29572
I20260808 04:45:38.005434 30201 tablet_replica.cc:330] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: stopping tablet replica
I20260808 04:45:38.005846 30201 raft_consensus.cc:2229] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:38.006258 30201 raft_consensus.cc:2258] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:38.009946 30201 ts_tablet_manager.cc:1905] T d1ed5cb57e13474b88ec9ce7a0ae5490 P e38c32386c3a4a9c9006ce7e3a25bb34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:38.013329 30197 tablet_replica.cc:330] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: stopping tablet replica
I20260808 04:45:38.013679 30197 raft_consensus.cc:2229] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:38.014014 30197 raft_consensus.cc:2258] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:38.016443 30197 ts_tablet_manager.cc:1905] T d1ed5cb57e13474b88ec9ce7a0ae5490 P fb69b300db304e049397e24ff1019a4a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 04:45:38.022921 29448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:35257 (error 108)
I20260808 04:45:38.023872 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 29757
I20260808 04:45:38.024063 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:38.023750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35257 (local address 127.0.0.1:51228)
0808 04:45:38.023911 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:38.023986 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 04:45:38.042402 29448 catalog_manager.cc:4584] TS f676f4a9007c420da68996abbb8f8e84 (127.22.2.129:35257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1ed5cb57e13474b88ec9ce7a0ae5490: Network error: Client connection negotiation failed: client connection to 127.22.2.129:35257: connect: Connection refused (error 111)
I20260808 04:45:38.051101 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 29947
I20260808 04:45:38.051765 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:38.051380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:43509 (local address 127.0.0.1:41886)
0808 04:45:38.051597 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:38.051656 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.130:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260808 04:45:38.065191 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 29424
2026-08-08T04:45:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1320 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 04:45:38.141214 22538 mini_oidc.cc:152] Starting JWKS server
I20260808 04:45:38.143101 22538 webserver.cc:460] Webserver started at https://127.0.0.1:43633/ using document root <none> and password file <none>
I20260808 04:45:38.143321 22538 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 04:45:38.143635 22538 webserver.cc:460] Webserver started at http://127.0.0.1:33413/ using document root <none> and password file <none>
2026-08-08T04:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:38Z Disabled control of system clock
I20260808 04:45:38.154215 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38227
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38227
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43633/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/testchainca.cert with env {}
W20260808 04:45:38.270062 30213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:38.270332 30213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:38.270419 30213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:38.274860 30213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:38.274915 30213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:38.274930 30213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:38.274947 30213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:38.274996 30213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 04:45:38.275018 30213 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 04:45:38.275048 30213 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43633/jwks.json
I20260808 04:45:38.280364 30213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43633/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:38.281462 30213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:38.282481 30213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:38.291519 30218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:38.291500 30224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:38.291502 30220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:38.291889 30213 server_base.cc:1034] running on GCE node
I20260808 04:45:38.292454 30213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:38.293763 30213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:38.294962 30213 hybrid_clock.cc:648] HybridClock initialized: now 1786164338294934 us; error 40 us; skew 500 ppm
I20260808 04:45:38.297116 30213 webserver.cc:460] Webserver started at http://127.22.2.190:44507/ using document root <none> and password file <none>
I20260808 04:45:38.297719 30213 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:38.297812 30213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:38.298050 30213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:38.300330 30213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "d6c602c32aa64baaa2a8bad421c1aff5"
format_stamp: "Formatted at 2026-08-08 04:45:38 on dist-test-slave-04bb"
server_key: "cfd384dc62a3489a61aef4c01c273293"
server_key_iv: "e7a9315e7ed7aa58686404c655358d38"
server_key_version: "encryptionkey@0"
I20260808 04:45:38.300968 30213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "d6c602c32aa64baaa2a8bad421c1aff5"
format_stamp: "Formatted at 2026-08-08 04:45:38 on dist-test-slave-04bb"
server_key: "cfd384dc62a3489a61aef4c01c273293"
server_key_iv: "e7a9315e7ed7aa58686404c655358d38"
server_key_version: "encryptionkey@0"
I20260808 04:45:38.304909 30213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:45:38.307457 30232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:38.308923 30213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:38.309063 30213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "d6c602c32aa64baaa2a8bad421c1aff5"
format_stamp: "Formatted at 2026-08-08 04:45:38 on dist-test-slave-04bb"
server_key: "cfd384dc62a3489a61aef4c01c273293"
server_key_iv: "e7a9315e7ed7aa58686404c655358d38"
server_key_version: "encryptionkey@0"
I20260808 04:45:38.309178 30213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:38.321732 30213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:38.322402 30213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:38.322634 30213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:38.332152 30213 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:38227
I20260808 04:45:38.332201 30310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:38227 every 8 connection(s)
I20260808 04:45:38.333266 30213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:38.336604 30312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:38.340803 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 30213
I20260808 04:45:38.340940 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:38.341261 22538 external_mini_cluster.cc:1428] Setting key e5f9aef6488962b04b84deea360d18b9
I20260808 04:45:38.342620 30312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5: Bootstrap starting.
I20260808 04:45:38.345218 30312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:38.346083 30312 log.cc:824] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:38.347930 30312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5: No bootstrap required, opened a new log
I20260808 04:45:38.351064 30318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:38.342780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58092 (local address 127.22.2.190:38227)
0808 04:45:38.343703 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:38.343723 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:38.343768 (+ 45us) server_negotiation.cc:407] Connection header received
0808 04:45:38.344455 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:38.344473 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:38.344988 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:38.345267 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:38.345684 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.346708 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.347614 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.348156 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.348337 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:38.349216 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:38.349228 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:38.349240 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:38.349619 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:38.349672 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:38.349692 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:38.349801 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:38.349942 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 04:45:38.350360 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":130,"threads_started":1}
I20260808 04:45:38.351548 30312 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c602c32aa64baaa2a8bad421c1aff5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38227 } }
I20260808 04:45:38.351804 30312 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:38.351883 30312 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c602c32aa64baaa2a8bad421c1aff5, State: Initialized, Role: FOLLOWER
I20260808 04:45:38.352408 30312 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c602c32aa64baaa2a8bad421c1aff5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38227 } }
I20260808 04:45:38.352586 30312 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:38.352651 30312 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:38.352759 30312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:38.354099 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:45:38.440769 30312 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c602c32aa64baaa2a8bad421c1aff5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38227 } }
I20260808 04:45:38.441388 30312 leader_election.cc:302] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6c602c32aa64baaa2a8bad421c1aff5; no voters:
I20260808 04:45:38.441942 30312 leader_election.cc:288] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:38.442129 30320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:38.442508 30320 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [term 1 LEADER]: Becoming Leader. State: Replica: d6c602c32aa64baaa2a8bad421c1aff5, State: Running, Role: LEADER
I20260808 04:45:38.442960 30320 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c602c32aa64baaa2a8bad421c1aff5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38227 } }
I20260808 04:45:38.443662 30312 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:38.445147 30321 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6c602c32aa64baaa2a8bad421c1aff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c602c32aa64baaa2a8bad421c1aff5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38227 } } }
I20260808 04:45:38.445294 30321 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:38.445616 30322 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6c602c32aa64baaa2a8bad421c1aff5. Latest consensus state: current_term: 1 leader_uuid: "d6c602c32aa64baaa2a8bad421c1aff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c602c32aa64baaa2a8bad421c1aff5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38227 } } }
I20260808 04:45:38.445719 30331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:38.445746 30322 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:38.448881 30331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:38.454468 30331 catalog_manager.cc:1347] Generated new cluster ID: c50e35bfe32d42f8a6bea36a4c70dda5
I20260808 04:45:38.454532 30331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:38.465059 30331 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:38.465972 30331 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:38.475400 30331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6c602c32aa64baaa2a8bad421c1aff5: Generated new TSK 0
I20260808 04:45:38.476048 30331 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 04:45:38.520080 30358 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 04:45:38.520291 30358 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 04:45:38.520892 30318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:38.499573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58094 (local address 127.22.2.190:38227)
0808 04:45:38.499718 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:38.499723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:38.499971 (+ 248us) server_negotiation.cc:407] Connection header received
0808 04:45:38.500187 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:38.500189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:38.500236 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:38.500342 (+ 106us) server_negotiation.cc:226] Negotiated authn=JWT
0808 04:45:38.500646 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.501232 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.502465 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.502679 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.502757 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:38.503123 (+ 366us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 04:45:38.520440 (+ 17317us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 04:45:38.520742 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 04:45:38.520994 30318 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 04:45:38.522343 30355 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:38.499498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38227 (local address 127.0.0.1:58094)
0808 04:45:38.499878 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:38.499894 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:38.500034 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:38.500375 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:38.500379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:38.500388 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 04:45:38.500541 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:38.500549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.501345 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.501349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:38.502345 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:38.502356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.502794 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.502798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:38.502895 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:38.502950 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 04:45:38.520747 (+ 17797us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 04:45:38.520992 (+ 245us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.2.190:38227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":123,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.2.190:38227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 04:45:38.523253 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 30213
2026-08-08T04:45:38Z chronyd exiting
I20260808 04:45:38.571964 22538 test_util.cc:175] -----------------------------------------------
I20260808 04:45:38.572054 22538 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786164310373382-22538-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (432 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 04:45:38.573246 22538 mini_oidc.cc:152] Starting JWKS server
I20260808 04:45:38.574195 22538 webserver.cc:460] Webserver started at https://127.0.0.1:44783/ using document root <none> and password file <none>
I20260808 04:45:38.574308 22538 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 04:45:38.574620 22538 webserver.cc:460] Webserver started at http://127.0.0.1:39175/ using document root <none> and password file <none>
2026-08-08T04:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:38Z Disabled control of system clock
I20260808 04:45:38.585137 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45765
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:45765
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44783/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/testchainca.cert with env {}
W20260808 04:45:38.705191 30366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:38.705446 30366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:38.705484 30366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:38.709561 30366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:38.709615 30366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:38.709630 30366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:38.709653 30366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:38.709668 30366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 04:45:38.709683 30366 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 04:45:38.709697 30366 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44783/jwks.json
I20260808 04:45:38.715139 30366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:45765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44783/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:38.716099 30366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:38.717180 30366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:38.726330 30376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:38.726341 30379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:38.726344 30382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:38.727029 30366 server_base.cc:1034] running on GCE node
I20260808 04:45:38.727588 30366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:38.728842 30366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:38.730062 30366 hybrid_clock.cc:648] HybridClock initialized: now 1786164338730016 us; error 62 us; skew 500 ppm
I20260808 04:45:38.732481 30366 webserver.cc:460] Webserver started at http://127.22.2.190:41733/ using document root <none> and password file <none>
I20260808 04:45:38.733151 30366 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:38.733220 30366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:38.733408 30366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:38.735456 30366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "bf7015e2843641e68210f585d0a9e1bd"
format_stamp: "Formatted at 2026-08-08 04:45:38 on dist-test-slave-04bb"
server_key: "3e3dcbec8e4c743851a5fef7a864cebc"
server_key_iv: "a349260caaeb346ab989b0daf1bb89b4"
server_key_version: "encryptionkey@0"
I20260808 04:45:38.736033 30366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "bf7015e2843641e68210f585d0a9e1bd"
format_stamp: "Formatted at 2026-08-08 04:45:38 on dist-test-slave-04bb"
server_key: "3e3dcbec8e4c743851a5fef7a864cebc"
server_key_iv: "a349260caaeb346ab989b0daf1bb89b4"
server_key_version: "encryptionkey@0"
I20260808 04:45:38.739897 30366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 04:45:38.742455 30389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:38.743944 30366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:38.744082 30366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "bf7015e2843641e68210f585d0a9e1bd"
format_stamp: "Formatted at 2026-08-08 04:45:38 on dist-test-slave-04bb"
server_key: "3e3dcbec8e4c743851a5fef7a864cebc"
server_key_iv: "a349260caaeb346ab989b0daf1bb89b4"
server_key_version: "encryptionkey@0"
I20260808 04:45:38.744215 30366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:38.761914 30366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:38.762547 30366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:38.762696 30366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:38.770105 30366 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:45765
I20260808 04:45:38.770183 30467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:45765 every 8 connection(s)
I20260808 04:45:38.771126 30366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:38.774418 30468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:38.779881 30468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd: Bootstrap starting.
I20260808 04:45:38.781100 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 30366
I20260808 04:45:38.781242 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:38.781522 22538 external_mini_cluster.cc:1428] Setting key 1417e1c6a4665e127b8fd4dd824ee496
I20260808 04:45:38.783057 30468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:38.784034 30468 log.cc:824] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:38.786005 30468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd: No bootstrap required, opened a new log
I20260808 04:45:38.789624 30468 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7015e2843641e68210f585d0a9e1bd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45765 } }
I20260808 04:45:38.789842 30468 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:38.789901 30468 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7015e2843641e68210f585d0a9e1bd, State: Initialized, Role: FOLLOWER
I20260808 04:45:38.790423 30468 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7015e2843641e68210f585d0a9e1bd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45765 } }
I20260808 04:45:38.790560 30468 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:38.790648 30468 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:38.790776 30468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:38.791424 30473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:38.782659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.22.2.190:45765)
0808 04:45:38.783387 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:38.783402 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:38.783439 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:45:38.784054 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:38.784070 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:38.784634 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:38.785037 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:38.785688 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.786759 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.787707 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.788227 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.788403 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:38.789375 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:38.789380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:38.789393 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:38.789779 (+ 386us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:38.789884 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:38.789921 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:38.790034 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:38.790231 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 04:45:38.790583 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":138,"threads_started":1}
I20260808 04:45:38.791676 30468 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7015e2843641e68210f585d0a9e1bd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45765 } }
I20260808 04:45:38.792654 30468 leader_election.cc:302] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf7015e2843641e68210f585d0a9e1bd; no voters:
I20260808 04:45:38.793036 30468 leader_election.cc:288] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:38.793111 30476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:38.793407 30476 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [term 1 LEADER]: Becoming Leader. State: Replica: bf7015e2843641e68210f585d0a9e1bd, State: Running, Role: LEADER
I20260808 04:45:38.793710 30476 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7015e2843641e68210f585d0a9e1bd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45765 } }
I20260808 04:45:38.794461 30468 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:38.795434 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:45:38.795697 30477 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf7015e2843641e68210f585d0a9e1bd. Latest consensus state: current_term: 1 leader_uuid: "bf7015e2843641e68210f585d0a9e1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7015e2843641e68210f585d0a9e1bd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45765 } } }
I20260808 04:45:38.795832 30477 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:38.796173 30476 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf7015e2843641e68210f585d0a9e1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7015e2843641e68210f585d0a9e1bd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45765 } } }
I20260808 04:45:38.796267 30476 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd [sys.catalog]: This master's current role is: LEADER
W20260808 04:45:38.799655 30496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:45:38.799757 30496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:45:38.799824 30487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:38.800475 30487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:38.806337 30487 catalog_manager.cc:1347] Generated new cluster ID: 9296b5040a0040778e081de149e84d8f
I20260808 04:45:38.806403 30487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:38.830619 30487 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:38.831548 30487 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:38.841691 30487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf7015e2843641e68210f585d0a9e1bd: Generated new TSK 0
I20260808 04:45:38.842382 30487 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 04:45:38.881589 30515 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 04:45:38.881702 30515 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 04:45:38.882145 30473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:38.858217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47388 (local address 127.22.2.190:45765)
0808 04:45:38.858373 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:38.858378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:38.858598 (+ 220us) server_negotiation.cc:407] Connection header received
0808 04:45:38.858764 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:38.858766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:38.858815 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:38.858906 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0808 04:45:38.859203 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.859735 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.861028 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.861267 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.861349 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:38.861726 (+ 377us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 04:45:38.881649 (+ 19923us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 04:45:38.881999 (+ 350us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47388: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 04:45:38.882236 30473 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47388: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 04:45:38.882336 30513 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:38.858153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45765 (local address 127.0.0.1:47388)
0808 04:45:38.858493 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:38.858509 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:38.858634 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:38.858931 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:38.858934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:38.858945 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0808 04:45:38.859064 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:38.859070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.859849 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.859853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:38.860903 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:38.860914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:38.861392 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:38.861398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:38.861494 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:38.861535 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 04:45:38.881929 (+ 20394us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 04:45:38.882138 (+ 209us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.2.190:45765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":131,"threads_started":1}
I20260808 04:45:38.883074 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 30366
2026-08-08T04:45:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (354 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 04:45:38.927340 22538 mini_oidc.cc:152] Starting JWKS server
I20260808 04:45:38.928221 22538 webserver.cc:460] Webserver started at https://127.0.0.1:35477/ using document root <none> and password file <none>
I20260808 04:45:38.928305 22538 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 04:45:38.928589 22538 webserver.cc:460] Webserver started at http://127.0.0.1:46249/ using document root <none> and password file <none>
2026-08-08T04:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:38Z Disabled control of system clock
I20260808 04:45:38.938710 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33893
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:33893
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35477/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 04:45:39.049710 30522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:39.049963 30522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:39.050001 30522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:39.054023 30522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:39.054076 30522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:39.054091 30522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:39.054107 30522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:39.054122 30522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 04:45:39.054138 30522 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 04:45:39.054154 30522 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35477/jwks.json
I20260808 04:45:39.058694 30522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:33893
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35477/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:39.059633 30522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:39.060966 30522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:39.069180 30538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:39.069181 30534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:39.069187 30533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:39.069447 30522 server_base.cc:1034] running on GCE node
I20260808 04:45:39.069996 30522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:39.071208 30522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:39.072400 30522 hybrid_clock.cc:648] HybridClock initialized: now 1786164339072376 us; error 47 us; skew 500 ppm
I20260808 04:45:39.074285 30522 webserver.cc:460] Webserver started at http://127.22.2.190:39579/ using document root <none> and password file <none>
I20260808 04:45:39.074853 30522 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:39.074944 30522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:39.075184 30522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:39.077750 30522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "9d3c498a1e2d4e74b18a520b0eb381f2"
format_stamp: "Formatted at 2026-08-08 04:45:39 on dist-test-slave-04bb"
server_key: "33600300359fd400de2b0fcd809014c6"
server_key_iv: "2884644f95714505f350a338b96c9502"
server_key_version: "encryptionkey@0"
I20260808 04:45:39.078394 30522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "9d3c498a1e2d4e74b18a520b0eb381f2"
format_stamp: "Formatted at 2026-08-08 04:45:39 on dist-test-slave-04bb"
server_key: "33600300359fd400de2b0fcd809014c6"
server_key_iv: "2884644f95714505f350a338b96c9502"
server_key_version: "encryptionkey@0"
I20260808 04:45:39.082348 30522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 04:45:39.084951 30544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:39.086225 30522 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:39.086382 30522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "9d3c498a1e2d4e74b18a520b0eb381f2"
format_stamp: "Formatted at 2026-08-08 04:45:39 on dist-test-slave-04bb"
server_key: "33600300359fd400de2b0fcd809014c6"
server_key_iv: "2884644f95714505f350a338b96c9502"
server_key_version: "encryptionkey@0"
I20260808 04:45:39.086496 30522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:39.107622 30522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:39.108294 30522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:39.108486 30522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:39.115471 30522 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:33893
I20260808 04:45:39.115499 30619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:33893 every 8 connection(s)
I20260808 04:45:39.116397 30522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:39.119532 30620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:39.124423 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 30522
I20260808 04:45:39.124573 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:39.124866 22538 external_mini_cluster.cc:1428] Setting key 194a292a1fb5fe2af40125e7aaba3eec
I20260808 04:45:39.125407 30620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2: Bootstrap starting.
I20260808 04:45:39.127945 30620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:39.128871 30620 log.cc:824] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:39.130748 30620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2: No bootstrap required, opened a new log
I20260808 04:45:39.134259 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:39.126313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.22.2.190:33893)
0808 04:45:39.126991 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:39.127008 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:39.127043 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:45:39.127663 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:39.127679 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:39.128168 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:39.128509 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:39.129030 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.130021 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.130936 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.131417 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.131592 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:39.132486 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:39.132491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:39.132503 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:39.132935 (+ 432us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:39.132988 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:39.133009 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:39.133123 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:39.133297 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 04:45:39.133548 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":154,"threads_started":1}
I20260808 04:45:39.134653 30620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3c498a1e2d4e74b18a520b0eb381f2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33893 } }
I20260808 04:45:39.134891 30620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:39.134970 30620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3c498a1e2d4e74b18a520b0eb381f2, State: Initialized, Role: FOLLOWER
I20260808 04:45:39.135442 30620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3c498a1e2d4e74b18a520b0eb381f2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33893 } }
I20260808 04:45:39.135598 30620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:39.135686 30620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:39.135807 30620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:39.136881 30620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3c498a1e2d4e74b18a520b0eb381f2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33893 } }
I20260808 04:45:39.137156 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:45:39.137264 30620 leader_election.cc:302] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d3c498a1e2d4e74b18a520b0eb381f2; no voters:
I20260808 04:45:39.137624 30620 leader_election.cc:288] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:39.137717 30630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:39.137893 30630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [term 1 LEADER]: Becoming Leader. State: Replica: 9d3c498a1e2d4e74b18a520b0eb381f2, State: Running, Role: LEADER
I20260808 04:45:39.138187 30630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3c498a1e2d4e74b18a520b0eb381f2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33893 } }
I20260808 04:45:39.138949 30620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:39.139434 30630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d3c498a1e2d4e74b18a520b0eb381f2. Latest consensus state: current_term: 1 leader_uuid: "9d3c498a1e2d4e74b18a520b0eb381f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3c498a1e2d4e74b18a520b0eb381f2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33893 } } }
I20260808 04:45:39.139549 30630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:39.139976 30633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d3c498a1e2d4e74b18a520b0eb381f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3c498a1e2d4e74b18a520b0eb381f2" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33893 } } }
I20260808 04:45:39.140141 30633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:39.140856 30640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:39.143625 30640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:39.148768 30640 catalog_manager.cc:1347] Generated new cluster ID: 3655aab4b93f49e3a56e6ba040424d0e
I20260808 04:45:39.148829 30640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:39.164492 30640 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:39.165669 30640 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:39.173722 30640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d3c498a1e2d4e74b18a520b0eb381f2: Generated new TSK 0
I20260808 04:45:39.174448 30640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:45:39.188825 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:39.183683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.22.2.190:33893)
0808 04:45:39.183812 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:39.183817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:39.184049 (+ 232us) server_negotiation.cc:407] Connection header received
0808 04:45:39.184206 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:39.184208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:39.184254 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:39.184342 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0808 04:45:39.184638 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.185162 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.186390 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.186617 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.186706 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:39.187080 (+ 374us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 04:45:39.188440 (+ 1360us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 04:45:39.188691 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56508: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 04:45:39.188885 30628 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56508: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 04:45:39.190941 30669 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:39.183621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33893 (local address 127.0.0.1:56508)
0808 04:45:39.183951 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:39.183966 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:39.184083 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:39.184365 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:39.184370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:39.184381 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0808 04:45:39.184499 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:39.184507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.185275 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.185278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:39.186254 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:39.186266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.186767 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.186772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:39.186871 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:39.186900 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 04:45:39.188631 (+ 1731us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 04:45:39.190767 (+ 2136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.2.190:33893: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":1923,"thread_start_us":134,"threads_started":1}
I20260808 04:45:39.191846 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 30522
E20260808 04:45:39.206326 30670 webserver.cc:533] Webserver: sslize failed:
E20260808 04:45:39.206401 30670 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T04:45:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (311 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T04:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:39Z Disabled control of system clock
I20260808 04:45:39.248890 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42767
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42767
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/testchainca.cert with env {}
W20260808 04:45:39.361608 30676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:39.361919 30676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:39.362010 30676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:39.366070 30676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:39.366163 30676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:39.366199 30676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:39.366253 30676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:39.366294 30676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 04:45:39.366331 30676 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 04:45:39.366364 30676 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 04:45:39.371552 30676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:39.372629 30676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:39.373620 30676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:39.382114 30684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:39.382171 30686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:39.382215 30683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:39.382453 30676 server_base.cc:1034] running on GCE node
I20260808 04:45:39.382995 30676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:39.384094 30676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:39.385254 30676 hybrid_clock.cc:648] HybridClock initialized: now 1786164339385230 us; error 49 us; skew 500 ppm
I20260808 04:45:39.386999 30676 webserver.cc:460] Webserver started at http://127.22.2.190:45685/ using document root <none> and password file <none>
I20260808 04:45:39.387573 30676 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:39.387637 30676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:39.387907 30676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:39.389932 30676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "a8c044a6af2a4ce299a3f616fee50f32"
format_stamp: "Formatted at 2026-08-08 04:45:39 on dist-test-slave-04bb"
server_key: "b75214f39990f52dc0d5b313fc5560e2"
server_key_iv: "104a2ae4c9fde608806c56bc2f36cf5f"
server_key_version: "encryptionkey@0"
I20260808 04:45:39.390479 30676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "a8c044a6af2a4ce299a3f616fee50f32"
format_stamp: "Formatted at 2026-08-08 04:45:39 on dist-test-slave-04bb"
server_key: "b75214f39990f52dc0d5b313fc5560e2"
server_key_iv: "104a2ae4c9fde608806c56bc2f36cf5f"
server_key_version: "encryptionkey@0"
I20260808 04:45:39.394055 30676 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:39.396565 30693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:39.397795 30676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:39.397922 30676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "a8c044a6af2a4ce299a3f616fee50f32"
format_stamp: "Formatted at 2026-08-08 04:45:39 on dist-test-slave-04bb"
server_key: "b75214f39990f52dc0d5b313fc5560e2"
server_key_iv: "104a2ae4c9fde608806c56bc2f36cf5f"
server_key_version: "encryptionkey@0"
I20260808 04:45:39.398022 30676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:39.417447 30676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:39.418103 30676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:39.418306 30676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:39.426038 30676 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42767
I20260808 04:45:39.426064 30782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42767 every 8 connection(s)
I20260808 04:45:39.427007 30676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:39.429970 30783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:39.434598 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 30676
I20260808 04:45:39.434725 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:39.435039 22538 external_mini_cluster.cc:1428] Setting key 9d783ed9b3badf07eaff9939d67f4ac8
I20260808 04:45:39.435966 30783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32: Bootstrap starting.
I20260808 04:45:39.438359 30783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:39.439275 30783 log.cc:824] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:39.441442 30783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32: No bootstrap required, opened a new log
I20260808 04:45:39.444820 30786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:39.436506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36006 (local address 127.22.2.190:42767)
0808 04:45:39.437177 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:39.437193 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:39.437228 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:45:39.437834 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:39.437850 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:39.438271 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:39.438592 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:39.438947 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.440148 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.441409 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.441919 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.442038 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:39.442979 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:39.442984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:39.442996 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:45:39.443410 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:39.443463 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:45:39.443483 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:39.443559 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:39.443740 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 04:45:39.444029 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":131,"threads_started":1}
I20260808 04:45:39.445595 30783 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c044a6af2a4ce299a3f616fee50f32" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42767 } }
I20260808 04:45:39.445866 30783 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:39.445958 30783 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c044a6af2a4ce299a3f616fee50f32, State: Initialized, Role: FOLLOWER
I20260808 04:45:39.446408 30783 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c044a6af2a4ce299a3f616fee50f32" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42767 } }
I20260808 04:45:39.446533 30783 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:39.446621 30783 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:39.446727 30783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:39.447609 30783 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c044a6af2a4ce299a3f616fee50f32" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42767 } }
I20260808 04:45:39.447738 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:45:39.447994 30783 leader_election.cc:302] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8c044a6af2a4ce299a3f616fee50f32; no voters:
I20260808 04:45:39.448313 30783 leader_election.cc:288] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:39.448451 30790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:39.448767 30790 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [term 1 LEADER]: Becoming Leader. State: Replica: a8c044a6af2a4ce299a3f616fee50f32, State: Running, Role: LEADER
I20260808 04:45:39.449102 30790 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c044a6af2a4ce299a3f616fee50f32" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42767 } }
I20260808 04:45:39.449613 30783 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:39.450299 30792 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8c044a6af2a4ce299a3f616fee50f32. Latest consensus state: current_term: 1 leader_uuid: "a8c044a6af2a4ce299a3f616fee50f32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c044a6af2a4ce299a3f616fee50f32" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42767 } } }
I20260808 04:45:39.450456 30792 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:39.450842 30801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:39.450835 30791 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8c044a6af2a4ce299a3f616fee50f32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c044a6af2a4ce299a3f616fee50f32" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42767 } } }
I20260808 04:45:39.451004 30791 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:39.454290 30801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:39.459451 30801 catalog_manager.cc:1347] Generated new cluster ID: d297e3f2d0974e1f9d152898cd25b822
I20260808 04:45:39.459510 30801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:39.482765 30801 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:39.483651 30801 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:39.488547 30801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8c044a6af2a4ce299a3f616fee50f32: Generated new TSK 0
I20260808 04:45:39.489365 30801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:45:39.606993 30786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:39.498159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.22.2.190:42767)
0808 04:45:39.498290 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:39.498294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:39.498452 (+ 158us) server_negotiation.cc:407] Connection header received
0808 04:45:39.498579 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:39.498583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:39.498639 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:39.498758 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0808 04:45:39.498980 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.499889 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.500782 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.501144 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.501262 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:39.501615 (+ 353us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 04:45:39.606598 (+104983us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 04:45:39.606872 (+ 274us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36020: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 04:45:39.607098 30786 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36020: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 04:45:39.607226 30831 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:39.498028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42767 (local address 127.0.0.1:36020)
0808 04:45:39.498367 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:39.498379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:39.498474 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:39.498763 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:39.498765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:39.498772 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0808 04:45:39.498877 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:39.498881 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.500004 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.500006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:39.500674 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:39.500683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:39.501340 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:39.501345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:39.501417 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:39.501440 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 04:45:39.606847 (+105407us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 04:45:39.607036 (+ 189us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.2.190:42767: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":118,"threads_started":1}
I20260808 04:45:39.608036 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 30676
2026-08-08T04:45:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (398 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (132 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:39 dist-test-slave-04bb krb5kdc[30849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:39 dist-test-slave-04bb krb5kdc[30849](info): set up 2 sockets
Aug 08 04:45:39 dist-test-slave-04bb krb5kdc[30849](info): commencing operation
krb5kdc: starting...
W20260808 04:45:41.110826 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:41 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:41.239480 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42349
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:41.352857 30867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:41.353159 30867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:41.353235 30867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:41.357386 30867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:41.357465 30867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:41.357501 30867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:41.357559 30867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:41.357599 30867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:41.362811 30867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:41.363874 30867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:41.364948 30867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:41.373790 30872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:41.373795 30875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:41.373870 30878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:41.374188 30867 server_base.cc:1034] running on GCE node
I20260808 04:45:41.374790 30867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:41.375833 30867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:41.376993 30867 hybrid_clock.cc:648] HybridClock initialized: now 1786164341376940 us; error 64 us; skew 500 ppm
Aug 08 04:45:41 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:41.379964 30867 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:41.381075 30867 webserver.cc:460] Webserver started at http://127.22.2.190:34835/ using document root <none> and password file <none>
I20260808 04:45:41.381666 30867 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:41.381762 30867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:41.381980 30867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:41.383998 30867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "8bbd9341dde0451db5f850ddc33683dc"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "4c65e3a909853740980f71aa4979205e"
server_key_iv: "1582c8249c146cc4dcdabb0555cdccbb"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.384624 30867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "8bbd9341dde0451db5f850ddc33683dc"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "4c65e3a909853740980f71aa4979205e"
server_key_iv: "1582c8249c146cc4dcdabb0555cdccbb"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.388417 30867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 04:45:41.390877 30886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:41.392135 30867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:41.392310 30867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "8bbd9341dde0451db5f850ddc33683dc"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "4c65e3a909853740980f71aa4979205e"
server_key_iv: "1582c8249c146cc4dcdabb0555cdccbb"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.392426 30867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:41.416380 30867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:41.419289 30867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:41.419502 30867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:41.426951 30867 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42349
I20260808 04:45:41.426981 30967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42349 every 8 connection(s)
I20260808 04:45:41.427935 30867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:41.430728 30969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:41.435760 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 30867
I20260808 04:45:41.435917 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:41.436189 22538 external_mini_cluster.cc:1428] Setting key 664fc98323af1d6ab2255b8063530a74
I20260808 04:45:41.436626 30969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc: Bootstrap starting.
I20260808 04:45:41.439177 30969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:41.439929 30969 log.cc:824] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:41.442180 30969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc: No bootstrap required, opened a new log
Aug 08 04:45:41 dist-test-slave-04bb krb5kdc[30849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:41.446635 30969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bbd9341dde0451db5f850ddc33683dc" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42349 } }
I20260808 04:45:41.446924 30969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:41.447031 30969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bbd9341dde0451db5f850ddc33683dc, State: Initialized, Role: FOLLOWER
I20260808 04:45:41.447582 30969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bbd9341dde0451db5f850ddc33683dc" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42349 } }
I20260808 04:45:41.447774 30969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:41.447871 30969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:41.447997 30969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:41.449115 30969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bbd9341dde0451db5f850ddc33683dc" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42349 } }
I20260808 04:45:41.449501 30969 leader_election.cc:302] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bbd9341dde0451db5f850ddc33683dc; no voters:
I20260808 04:45:41.449870 30969 leader_election.cc:288] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:41.449973 30977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:41.450228 30977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [term 1 LEADER]: Becoming Leader. State: Replica: 8bbd9341dde0451db5f850ddc33683dc, State: Running, Role: LEADER
I20260808 04:45:41.450563 30977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bbd9341dde0451db5f850ddc33683dc" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42349 } }
I20260808 04:45:41.451188 30969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:41.452373 30980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bbd9341dde0451db5f850ddc33683dc. Latest consensus state: current_term: 1 leader_uuid: "8bbd9341dde0451db5f850ddc33683dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bbd9341dde0451db5f850ddc33683dc" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42349 } } }
I20260808 04:45:41.452517 30980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:41.452373 30979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bbd9341dde0451db5f850ddc33683dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bbd9341dde0451db5f850ddc33683dc" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42349 } } }
I20260808 04:45:41.452602 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:41.437704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.22.2.190:42349)
0808 04:45:41.438221 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:41.438238 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:41.438274 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:45:41.438893 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:41.438909 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:41.439263 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:41.439622 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:41.440505 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.441968 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.442958 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.443654 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.443827 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:41.446569 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:41.446575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:41.446591 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:41.446623 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:41.448822 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:41.449324 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:41.449330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:41.449336 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:41.449476 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:41.449774 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:41.449780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:41.449783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:41.450329 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:41.451098 (+ 769us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:41.451251 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 04:45:41.451551 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":155,"threads_started":1}
I20260808 04:45:41.452641 30979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:41.453209 30987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:41.455965 30987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:41.461155 30987 catalog_manager.cc:1347] Generated new cluster ID: ef236f4daf504e2eaed14a1e15dc1e8a
I20260808 04:45:41.461221 30987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:41.482632 30987 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:41.483829 30987 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:41.494650 30987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bbd9341dde0451db5f850ddc33683dc: Generated new TSK 0
I20260808 04:45:41.495446 30987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:41.535540 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42349
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:41.648090 31008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:41.648320 31008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:41.648360 31008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:41.652421 31008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:41.652494 31008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:41.652592 31008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:41.657855 31008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:41.658773 31008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:41.659709 31008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:41.669261 31017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:41.669282 31014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:41.669417 31008 server_base.cc:1034] running on GCE node
W20260808 04:45:41.669298 31013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:41.669843 31008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:41.670397 31008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:41.671592 31008 hybrid_clock.cc:648] HybridClock initialized: now 1786164341671570 us; error 31 us; skew 500 ppm
Aug 08 04:45:41 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:41.674400 31008 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:41.675287 31008 webserver.cc:460] Webserver started at http://127.22.2.129:33839/ using document root <none> and password file <none>
I20260808 04:45:41.675865 31008 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:41.675962 31008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:41.676198 31008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:41.678205 31008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "79100905ec8a4a45b98e7470ddcf335f"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "15baab883fd3c6df89d0f24fbe9c8aef"
server_key_iv: "19541d74698f5e0ddcc08f8bb1ac13bd"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.678795 31008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "79100905ec8a4a45b98e7470ddcf335f"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "15baab883fd3c6df89d0f24fbe9c8aef"
server_key_iv: "19541d74698f5e0ddcc08f8bb1ac13bd"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.682513 31008 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:41.684957 31030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:41.686151 31008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:41.686296 31008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "79100905ec8a4a45b98e7470ddcf335f"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "15baab883fd3c6df89d0f24fbe9c8aef"
server_key_iv: "19541d74698f5e0ddcc08f8bb1ac13bd"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.686416 31008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:41.718739 31008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:41.721702 31008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:41.721917 31008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:41.722550 31008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:41.723418 31008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:41.723493 31008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:41.723567 31008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:41.723610 31008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:41.732666 31008 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:45383
I20260808 04:45:41.732700 31204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:45383 every 8 connection(s)
I20260808 04:45:41.733616 31008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:41.742606 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 31008
I20260808 04:45:41.742743 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:41.743074 22538 external_mini_cluster.cc:1428] Setting key 3f9081a215f9ecf5a3fad86594b6a0c5
Aug 08 04:45:41 dist-test-slave-04bb krb5kdc[30849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:41.749511 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:41.735599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:40737 (local address 127.22.2.190:42349)
0808 04:45:41.735743 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:41.735747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:41.736621 (+ 874us) server_negotiation.cc:407] Connection header received
0808 04:45:41.738027 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:41.738030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:41.738088 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:41.738198 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:41.739384 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.739923 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.741517 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.741881 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.741988 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:41.745459 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:41.745465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:41.745471 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:41.745505 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:41.747167 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:41.747813 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:41.747815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:41.747816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:41.747903 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:41.748315 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:41.748317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:41.748318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:41.748483 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:41.748627 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:41.749209 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 04:45:41.749343 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:45:41.750257 31211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:41.735991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42349 (local address 127.22.2.129:40737)
0808 04:45:41.736448 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:41.736492 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:41.737812 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:41.738392 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:41.738399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:41.738748 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:41.739222 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:41.739235 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.740084 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.740087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:41.741385 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:41.741396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.742028 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.742032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:41.742236 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:41.743559 (+ 1323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:41.743579 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:41.745283 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:41.747311 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:41.747319 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:41.747334 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:41.747699 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:41.747995 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:41.748001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:41.748004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:41.748194 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:41.748631 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:41.748639 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:41.748959 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 04:45:41.749146 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 04:45:41.749453 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":106,"threads_started":1}
I20260808 04:45:41.751473 31207 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42349
I20260808 04:45:41.751729 31207 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:41.752213 31207 heartbeater.cc:511] Master 127.22.2.190:42349 requested a full tablet report, sending...
I20260808 04:45:41.754058 30918 ts_manager.cc:194] Registered new tserver with Master: 79100905ec8a4a45b98e7470ddcf335f (127.22.2.129:45383)
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:45:41.755569 30918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:40737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:41.773622 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42349
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:41.887583 31216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:41.887840 31216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:41.887883 31216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:41.891943 31216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:41.891999 31216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:41.892055 31216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:41.897238 31216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:41.898196 31216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:41.899224 31216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:41.908708 31226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:41.908728 31223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:41.908906 31216 server_base.cc:1034] running on GCE node
W20260808 04:45:41.909056 31228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:41.909461 31216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:41.910080 31216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:41.911267 31216 hybrid_clock.cc:648] HybridClock initialized: now 1786164341911241 us; error 36 us; skew 500 ppm
Aug 08 04:45:41 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:41.913976 31216 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:41.914877 31216 webserver.cc:460] Webserver started at http://127.22.2.130:34571/ using document root <none> and password file <none>
I20260808 04:45:41.915453 31216 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:41.915544 31216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:41.915786 31216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:41.917841 31216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "811a5969dc344d18993c0c9a0104f76b"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "32eb7e34fa7e9bf39969d4769090a0eb"
server_key_iv: "049b4315a6d8ff77ba6723cd681fc1f5"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.918401 31216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "811a5969dc344d18993c0c9a0104f76b"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "32eb7e34fa7e9bf39969d4769090a0eb"
server_key_iv: "049b4315a6d8ff77ba6723cd681fc1f5"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.922219 31216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:45:41.924757 31237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:41.925992 31216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:41.926091 31216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "811a5969dc344d18993c0c9a0104f76b"
format_stamp: "Formatted at 2026-08-08 04:45:41 on dist-test-slave-04bb"
server_key: "32eb7e34fa7e9bf39969d4769090a0eb"
server_key_iv: "049b4315a6d8ff77ba6723cd681fc1f5"
server_key_version: "encryptionkey@0"
I20260808 04:45:41.926234 31216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:41.941505 31216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:41.944702 31216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:41.944911 31216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:41.945531 31216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:41.946408 31216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:41.946457 31216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:41.946497 31216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:41.946563 31216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:41.956131 31216 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:44455
I20260808 04:45:41.956200 31401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:44455 every 8 connection(s)
I20260808 04:45:41.957126 31216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:41.960137 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 31216
I20260808 04:45:41.960263 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:41.960649 22538 external_mini_cluster.cc:1428] Setting key 18c1541ed054b1d9b343fe5cbaba8ac1
Aug 08 04:45:41 dist-test-slave-04bb krb5kdc[30849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:41.972674 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:41.958953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:57199 (local address 127.22.2.190:42349)
0808 04:45:41.959086 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:41.959091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:41.959864 (+ 773us) server_negotiation.cc:407] Connection header received
0808 04:45:41.961614 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:41.961617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:41.961676 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:41.961764 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:41.963241 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.963775 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.965206 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.965440 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.965549 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:41.968663 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:41.968667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:41.968669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:41.968693 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:41.970209 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:41.970907 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:41.970910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:41.970911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:41.970986 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:41.971440 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:41.971442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:41.971443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:41.971610 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:41.971682 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:41.972400 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 04:45:41.972497 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:45:41.973735 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:41.959297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42349 (local address 127.22.2.130:57199)
0808 04:45:41.959716 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:41.959758 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:41.961397 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:41.961939 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:41.961946 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:41.962372 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:41.963055 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:41.963079 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.963910 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.963917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:41.965073 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:41.965084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:41.965600 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:41.965604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:41.965775 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:41.966407 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:41.966425 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:41.968465 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:41.970355 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:41.970365 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:41.970383 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:41.970792 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:41.971090 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:41.971098 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:41.971111 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:41.971320 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:41.971722 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:41.971729 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:41.972086 (+ 357us) client_negotiation.cc:769] Sending connection context
0808 04:45:41.972355 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 04:45:41.972747 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260808 04:45:41.974961 31403 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42349
I20260808 04:45:41.975219 31403 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:45:41.975857 31403 heartbeater.cc:511] Master 127.22.2.190:42349 requested a full tablet report, sending...
I20260808 04:45:41.976963 30918 ts_manager.cc:194] Registered new tserver with Master: 811a5969dc344d18993c0c9a0104f76b (127.22.2.130:44455)
I20260808 04:45:41.977715 30918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:57199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:41.992022 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42349
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:42.114132 31415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:42.114392 31415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:42.114501 31415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:42.118557 31415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:42.118634 31415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:42.118714 31415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:42.123822 31415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:42.124913 31415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:42.125919 31415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:42.135898 31420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:42.135912 31421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:42.135911 31424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:42.136089 31415 server_base.cc:1034] running on GCE node
I20260808 04:45:42.136682 31415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:42.137338 31415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:42.138527 31415 hybrid_clock.cc:648] HybridClock initialized: now 1786164342138518 us; error 32 us; skew 500 ppm
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:42.141233 31415 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:42.142136 31415 webserver.cc:460] Webserver started at http://127.22.2.131:40123/ using document root <none> and password file <none>
I20260808 04:45:42.142714 31415 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:42.142831 31415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:42.143067 31415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:42.145126 31415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "63f18f4ba79048e4a2cdebdc23a394ce"
format_stamp: "Formatted at 2026-08-08 04:45:42 on dist-test-slave-04bb"
server_key: "ee0fbe16bd2c440d5052df220827e6b1"
server_key_iv: "841824b183bf0431bdde34b38cd8ddef"
server_key_version: "encryptionkey@0"
I20260808 04:45:42.145687 31415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "63f18f4ba79048e4a2cdebdc23a394ce"
format_stamp: "Formatted at 2026-08-08 04:45:42 on dist-test-slave-04bb"
server_key: "ee0fbe16bd2c440d5052df220827e6b1"
server_key_iv: "841824b183bf0431bdde34b38cd8ddef"
server_key_version: "encryptionkey@0"
I20260808 04:45:42.149372 31415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:45:42.151750 31432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:42.153157 31415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:45:42.153263 31415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "63f18f4ba79048e4a2cdebdc23a394ce"
format_stamp: "Formatted at 2026-08-08 04:45:42 on dist-test-slave-04bb"
server_key: "ee0fbe16bd2c440d5052df220827e6b1"
server_key_iv: "841824b183bf0431bdde34b38cd8ddef"
server_key_version: "encryptionkey@0"
I20260808 04:45:42.153436 31415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:42.188414 31415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:42.191732 31415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:42.191946 31415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:42.192632 31415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:42.193483 31415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:42.193532 31415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:42.193608 31415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:42.193648 31415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:42.203383 31415 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:35281
I20260808 04:45:42.203415 31619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:35281 every 8 connection(s)
I20260808 04:45:42.204299 31415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:42.209405 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 31415
I20260808 04:45:42.209540 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:42.209870 22538 external_mini_cluster.cc:1428] Setting key c425943c97066e277a78f508220dcc9b
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:42.218644 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:42.206242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:40013 (local address 127.22.2.190:42349)
0808 04:45:42.206388 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:42.206392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:42.207207 (+ 815us) server_negotiation.cc:407] Connection header received
0808 04:45:42.208661 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:42.208664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:42.208713 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:42.208787 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:42.210334 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.210938 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.212489 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.212768 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.212861 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:42.215438 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:42.215440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:42.215442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:42.215463 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:42.216908 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:42.217388 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:42.217391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:42.217392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:42.217463 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:42.217759 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:42.217761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:42.217762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:42.217939 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:42.218016 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:42.218441 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 04:45:42.218530 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 04:45:42.219435 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:42.206612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42349 (local address 127.22.2.131:40013)
0808 04:45:42.207058 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:42.207098 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:42.208388 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:42.208950 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:42.208958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:42.209481 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:42.210149 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:42.210169 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.211081 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.211085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:42.212368 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:42.212379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.212917 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.212921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:42.213063 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:42.213772 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:42.213790 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:42.215293 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:42.217012 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:42.217017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:42.217027 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:42.217295 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:42.217540 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:42.217542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:42.217544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:42.217679 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:42.218025 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:42.218030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:42.218245 (+ 215us) client_negotiation.cc:769] Sending connection context
0808 04:45:42.218395 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 04:45:42.218627 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":111,"threads_started":1}
I20260808 04:45:42.220587 31620 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42349
I20260808 04:45:42.220829 31620 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:42.221310 31620 heartbeater.cc:511] Master 127.22.2.190:42349 requested a full tablet report, sending...
I20260808 04:45:42.222335 30918 ts_manager.cc:194] Registered new tserver with Master: 63f18f4ba79048e4a2cdebdc23a394ce (127.22.2.131:35281)
I20260808 04:45:42.222863 30918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:40013
I20260808 04:45:42.224318 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/test-admin.keytab.
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:42.241092 22538 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 04:45:42.243978 31628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:42.242455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.22.2.129:45383)
0808 04:45:42.242752 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:42.242767 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:42.242792 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:45:42.242843 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:42.242847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:42.243025 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:42.243150 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:42.243787 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37658: BlockingRecv error: recv got EOF from 127.0.0.1:37658 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":97,"threads_started":1}
W20260808 04:45:42.244040 22538 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5cc' of type FILE
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:42.245123 22538 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:42.246022 22538 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:42.255251 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:42.246824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37668 (local address 127.22.2.129:45383)
0808 04:45:42.246958 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:42.246962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:42.246975 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:45:42.247078 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:42.247080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:42.247119 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:42.247184 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:42.247680 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.248458 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.249286 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.249642 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.249723 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:42.251962 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:42.251970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:42.251983 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:42.252004 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:42.253577 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:42.254021 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:42.254026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:42.254027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:42.254100 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:42.254365 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:42.254367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:42.254369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:42.254584 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:42.254728 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:42.254944 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 04:45:42.255121 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 04:45:42.259403 31628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:42.258289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.22.2.129:45383)
0808 04:45:42.258429 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:42.258433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:42.258564 (+ 131us) server_negotiation.cc:407] Connection header received
0808 04:45:42.258735 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:42.258738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:42.258782 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:42.258876 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:42.259300 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37676: BlockingRecv error: recv got EOF from 127.0.0.1:37676 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 04:45:42.259526 22538 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5cc' of type FILE
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:42.260607 22538 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:42.261539 22538 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:42.269667 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:42.262342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.22.2.129:45383)
0808 04:45:42.262476 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:42.262479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:42.262493 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:45:42.262597 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:42.262599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:42.262633 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:42.262704 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:42.263180 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.263639 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.264378 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.264714 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.264802 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:42.266885 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:42.266888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:42.266890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:42.266912 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:42.268190 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:42.268611 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:42.268613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:42.268615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:42.268689 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:42.268919 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:42.268921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:42.268922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:42.269063 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:42.269155 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:42.269382 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 04:45:42.269532 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 04:45:42.272105 31628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:42.271258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.22.2.129:45383)
0808 04:45:42.271407 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:42.271411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:42.271478 (+ 67us) server_negotiation.cc:407] Connection header received
0808 04:45:42.271560 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:42.271562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:42.271594 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:42.271654 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:42.272016 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37696: BlockingRecv error: recv got EOF from 127.0.0.1:37696 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260808 04:45:42.272217 22538 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786164310373382-22538-0/krb5kdc/krb5cc' of type FILE
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:42.273200 22538 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:42.274056 22538 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[30849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:45:42.282272 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:42.274823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.22.2.129:45383)
0808 04:45:42.274950 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:42.274953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:42.274992 (+ 39us) server_negotiation.cc:407] Connection header received
0808 04:45:42.275095 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:42.275097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:42.275127 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:42.275236 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:42.275957 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.276495 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.277301 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:42.277500 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:42.277581 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:42.279611 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:42.279614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:42.279615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:42.279636 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:42.280871 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:42.281259 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:42.281262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:42.281263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:42.281338 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:42.281567 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:42.281569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:42.281570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:42.281708 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:42.281813 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:42.281997 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 04:45:42.282113 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 04:45:42.283319 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 31008
I20260808 04:45:42.295284 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 31216
I20260808 04:45:42.303544 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 31415
I20260808 04:45:42.321496 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 30867
2026-08-08T04:45:42Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2447 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[31653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[31653](info): set up 2 sockets
Aug 08 04:45:42 dist-test-slave-04bb krb5kdc[31653](info): commencing operation
krb5kdc: starting...
W20260808 04:45:43.572127 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:43 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.190.keytab.
Entry for principal oryx/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.190.keytab.
I20260808 04:45:43.699533 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38987
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.190.keytab
--principal=oryx/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:43.814658 31674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:43.814957 31674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:43.815035 31674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:43.819029 31674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:43.819123 31674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:43.819162 31674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:43.819198 31674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:43.819231 31674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:43.824558 31674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.190.keytab
--principal=oryx/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:43.825651 31674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:43.826719 31674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:43.835677 31683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:43.835654 31682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:43.835654 31686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:43.836081 31674 server_base.cc:1034] running on GCE node
I20260808 04:45:43.836704 31674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:43.837829 31674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:43.839020 31674 hybrid_clock.cc:648] HybridClock initialized: now 1786164343838991 us; error 42 us; skew 500 ppm
Aug 08 04:45:43 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164343, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:43.841851 31674 init.cc:377] Logged in from keytab as oryx/127.22.2.190@KRBTEST.COM (short username oryx)
I20260808 04:45:43.842728 31674 webserver.cc:460] Webserver started at http://127.22.2.190:45431/ using document root <none> and password file <none>
I20260808 04:45:43.843255 31674 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:43.843318 31674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:43.843482 31674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:43.845571 31674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "d2416c14ab864738b27a3a0891a4b474"
format_stamp: "Formatted at 2026-08-08 04:45:43 on dist-test-slave-04bb"
server_key: "82b4d9e44c6ed97e1f5424ca1049c7d6"
server_key_iv: "e46654b1f9c7bb807972b1002e8af4d3"
server_key_version: "encryptionkey@0"
I20260808 04:45:43.846088 31674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "d2416c14ab864738b27a3a0891a4b474"
format_stamp: "Formatted at 2026-08-08 04:45:43 on dist-test-slave-04bb"
server_key: "82b4d9e44c6ed97e1f5424ca1049c7d6"
server_key_iv: "e46654b1f9c7bb807972b1002e8af4d3"
server_key_version: "encryptionkey@0"
I20260808 04:45:43.849929 31674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:45:43.852284 31697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:43.853561 31674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:43.853664 31674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "d2416c14ab864738b27a3a0891a4b474"
format_stamp: "Formatted at 2026-08-08 04:45:43 on dist-test-slave-04bb"
server_key: "82b4d9e44c6ed97e1f5424ca1049c7d6"
server_key_iv: "e46654b1f9c7bb807972b1002e8af4d3"
server_key_version: "encryptionkey@0"
I20260808 04:45:43.853755 31674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:43.870894 31674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:43.873780 31674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:43.873950 31674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:43.881615 31674 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:38987
I20260808 04:45:43.881916 31774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:38987 every 8 connection(s)
I20260808 04:45:43.882655 31674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:43.885545 31775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:43.886044 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 31674
I20260808 04:45:43.886193 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:43.886497 22538 external_mini_cluster.cc:1428] Setting key a89ef3ce6644f354357e0ee03a63edfc
I20260808 04:45:43.893114 31775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: Bootstrap starting.
I20260808 04:45:43.895358 31775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: Neither blocks nor log segments found. Creating new log.
Aug 08 04:45:43 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:43.896155 31775 log.cc:824] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:43.898110 31775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: No bootstrap required, opened a new log
I20260808 04:45:43.901794 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:43.887986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.22.2.190:38987)
0808 04:45:43.888430 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:43.888449 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:43.888499 (+ 50us) server_negotiation.cc:407] Connection header received
0808 04:45:43.889301 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:43.889324 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:43.889773 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:43.890136 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:43.890835 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:43.892336 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:43.893338 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:43.893843 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:43.894031 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:43.896509 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:43.896514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:43.896570 (+ 56us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:43.896610 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:43.898461 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:43.898981 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:43.898987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:43.898992 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:43.899119 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:43.899468 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:43.899471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:43.899472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:43.899950 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:43.900143 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:43.900616 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 04:45:43.900993 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":109,"threads_started":1}
I20260808 04:45:43.902179 31775 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } }
I20260808 04:45:43.902434 31775 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:43.902513 31775 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2416c14ab864738b27a3a0891a4b474, State: Initialized, Role: FOLLOWER
I20260808 04:45:43.903012 31775 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } }
I20260808 04:45:43.903170 31775 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:43.903259 31775 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:43.903384 31775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:43.904276 31775 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } }
I20260808 04:45:43.904682 31775 leader_election.cc:302] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2416c14ab864738b27a3a0891a4b474; no voters:
I20260808 04:45:43.905014 31775 leader_election.cc:288] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:43.905154 31789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:43.905542 31789 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 1 LEADER]: Becoming Leader. State: Replica: d2416c14ab864738b27a3a0891a4b474, State: Running, Role: LEADER
I20260808 04:45:43.905853 31789 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } }
I20260808 04:45:43.906319 31775 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:43.907558 31790 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } }
I20260808 04:45:43.907706 31790 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:43.908113 31791 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2416c14ab864738b27a3a0891a4b474. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } }
I20260808 04:45:43.908149 31801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:43.908298 31791 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:43.911824 31801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:43.920583 31801 catalog_manager.cc:1347] Generated new cluster ID: e84031d74e684823b9b396333083bff1
I20260808 04:45:43.920657 31801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:43.926605 31801 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:43.927807 31801 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:43.933503 31801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: Generated new TSK 0
I20260808 04:45:43.934221 31801 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
I20260808 04:45:43.964697 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38987
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:44.079792 31818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:44.080058 31818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:44.080134 31818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:44.084126 31818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:44.084210 31818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:44.084306 31818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:44.089442 31818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:44.090555 31818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:44.091627 31818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:44.103324 31825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:44.103295 31828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:44.103287 31826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:44.103709 31818 server_base.cc:1034] running on GCE node
I20260808 04:45:44.104148 31818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:44.104815 31818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:44.105973 31818 hybrid_clock.cc:648] HybridClock initialized: now 1786164344105941 us; error 44 us; skew 500 ppm
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:44.108985 31818 init.cc:377] Logged in from keytab as oryx/127.22.2.129@KRBTEST.COM (short username oryx)
I20260808 04:45:44.109764 31818 webserver.cc:460] Webserver started at http://127.22.2.129:35553/ using document root <none> and password file <none>
I20260808 04:45:44.110283 31818 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:44.110347 31818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:44.110512 31818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:44.112597 31818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "b8e130596245467ab759328b94fd2ed2"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "5db7ef0866762ef7c7714968aae19fa8"
server_key_iv: "3f0b7e0e9431e8695bdee628bf6cc65e"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.113121 31818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "b8e130596245467ab759328b94fd2ed2"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "5db7ef0866762ef7c7714968aae19fa8"
server_key_iv: "3f0b7e0e9431e8695bdee628bf6cc65e"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.116700 31818 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:44.118952 31837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.120162 31818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:44.120316 31818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "b8e130596245467ab759328b94fd2ed2"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "5db7ef0866762ef7c7714968aae19fa8"
server_key_iv: "3f0b7e0e9431e8695bdee628bf6cc65e"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.120426 31818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:44.134521 31818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:44.137720 31818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:44.137949 31818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:44.138617 31818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:44.139554 31818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:44.139633 31818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.139712 31818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:44.139773 31818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.149524 31818 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:45035
I20260808 04:45:44.149549 32020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:45035 every 8 connection(s)
I20260808 04:45:44.150475 31818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:44.151547 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 31818
I20260808 04:45:44.151672 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:44.152006 22538 external_mini_cluster.cc:1428] Setting key 779dc5224c5c04dded5b634280cbb582
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.130@KRBTEST.COM" created.
I20260808 04:45:44.167621 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.152699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34967 (local address 127.22.2.190:38987)
0808 04:45:44.152883 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.152888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.153993 (+ 1105us) server_negotiation.cc:407] Connection header received
0808 04:45:44.156003 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.156009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.156077 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.156227 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.157496 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.158508 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.159834 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.160141 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.160243 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.163450 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.163454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.163456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.163482 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.165217 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.165772 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.165776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.165778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.165861 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.166267 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.166270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.166271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.166552 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.166659 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.167205 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.167360 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 04:45:44.168207 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.153190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:34967)
0808 04:45:44.153789 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.153867 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.155722 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.156355 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.156361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.156786 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.157327 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.157340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.158676 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.158680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.159699 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.159710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.160344 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.160352 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.160637 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.161367 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.161395 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.163261 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.165353 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.165359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.165374 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.165641 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.165962 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.165965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.165967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.166112 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.166691 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.166696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.166934 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.167088 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.167344 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":152,"threads_started":1}
I20260808 04:45:44.169617 32024 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38987
I20260808 04:45:44.169884 32024 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:44.170500 32024 heartbeater.cc:511] Master 127.22.2.190:38987 requested a full tablet report, sending...
I20260808 04:45:44.172406 31717 ts_manager.cc:194] Registered new tserver with Master: b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035)
I20260808 04:45:44.174132 31717 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:34967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
I20260808 04:45:44.184356 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38987
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:44.298638 32031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:44.298884 32031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:44.298926 32031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:44.302984 32031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:44.303040 32031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:44.303097 32031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:44.308557 32031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:44.309535 32031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:44.310629 32031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:44.320246 32037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:44.320262 32039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:44.320246 32042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:44.320446 32031 server_base.cc:1034] running on GCE node
I20260808 04:45:44.320925 32031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:44.321591 32031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:44.322749 32031 hybrid_clock.cc:648] HybridClock initialized: now 1786164344322714 us; error 47 us; skew 500 ppm
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:44.325675 32031 init.cc:377] Logged in from keytab as oryx/127.22.2.130@KRBTEST.COM (short username oryx)
I20260808 04:45:44.326579 32031 webserver.cc:460] Webserver started at http://127.22.2.130:44519/ using document root <none> and password file <none>
I20260808 04:45:44.327164 32031 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:44.327258 32031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:44.327512 32031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:44.329602 32031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "d8c1979ae5fb4ef0a97987256b08aa2c"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "69070d4bc0462035b8288c71bceae6e1"
server_key_iv: "67080a2a00af946a509b6b7b75c52e18"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.330185 32031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "d8c1979ae5fb4ef0a97987256b08aa2c"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "69070d4bc0462035b8288c71bceae6e1"
server_key_iv: "67080a2a00af946a509b6b7b75c52e18"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.334059 32031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:44.336593 32052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.337807 32031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:44.337945 32031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "d8c1979ae5fb4ef0a97987256b08aa2c"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "69070d4bc0462035b8288c71bceae6e1"
server_key_iv: "67080a2a00af946a509b6b7b75c52e18"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.338059 32031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:44.375119 32031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:44.378222 32031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:44.378451 32031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:44.379079 32031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:44.379967 32031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:44.380039 32031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.380113 32031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:44.380155 32031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.389381 32031 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:46229
I20260808 04:45:44.389405 32210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:46229 every 8 connection(s)
I20260808 04:45:44.390282 32031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:44.391505 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 32031
I20260808 04:45:44.391650 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:44.392010 22538 external_mini_cluster.cc:1428] Setting key 432d2761ea6c0a1f9202a65b96c0cccb
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.131@KRBTEST.COM" created.
I20260808 04:45:44.406435 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.392233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:55543 (local address 127.22.2.190:38987)
0808 04:45:44.392383 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.392389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.393421 (+ 1032us) server_negotiation.cc:407] Connection header received
0808 04:45:44.394855 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.394859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.394916 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.395016 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.396620 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.397172 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.398998 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.399239 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.399321 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.402379 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.402384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.402390 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.402428 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.404066 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.404825 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.404827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.404829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.404906 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.405217 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.405220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.405221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.405401 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.405477 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.406058 (+ 581us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.406167 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:45:44.407541 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.392740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:55543)
0808 04:45:44.393279 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.393335 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.394624 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.395202 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.395210 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.395711 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.396419 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.396442 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.397301 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.397308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.398854 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.398870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.399404 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.399408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.399645 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.400373 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.400401 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.402196 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.404267 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.404272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.404284 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.404705 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.404989 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.404991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.404993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.405127 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.405491 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.405495 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.405836 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.405994 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.406374 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":116,"threads_started":1}
I20260808 04:45:44.408890 32211 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38987
I20260808 04:45:44.409144 32211 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:44.409649 32211 heartbeater.cc:511] Master 127.22.2.190:38987 requested a full tablet report, sending...
I20260808 04:45:44.410813 31717 ts_manager.cc:194] Registered new tserver with Master: d8c1979ae5fb4ef0a97987256b08aa2c (127.22.2.130:46229)
I20260808 04:45:44.411700 31717 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:55543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
I20260808 04:45:44.423465 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38987
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:44.536702 32216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:44.536948 32216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:44.536990 32216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:44.541030 32216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:44.541086 32216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:44.541167 32216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:44.546473 32216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:44.547497 32216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:44.548609 32216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:44.558315 32227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:44.558334 32225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:44.558338 32224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:44.558723 32216 server_base.cc:1034] running on GCE node
I20260808 04:45:44.559131 32216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:44.559791 32216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:44.560992 32216 hybrid_clock.cc:648] HybridClock initialized: now 1786164344560961 us; error 46 us; skew 500 ppm
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:44.564056 32216 init.cc:377] Logged in from keytab as oryx/127.22.2.131@KRBTEST.COM (short username oryx)
I20260808 04:45:44.565091 32216 webserver.cc:460] Webserver started at http://127.22.2.131:40503/ using document root <none> and password file <none>
I20260808 04:45:44.565685 32216 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:44.565788 32216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:44.566031 32216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:44.568122 32216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "de3aeaed68624f448b39563148adfcbe"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "4c985d522278828e7ef46720d10e3bf7"
server_key_iv: "93bbaf7d096859a6b0ced446e89c3f6d"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.568771 32216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "de3aeaed68624f448b39563148adfcbe"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "4c985d522278828e7ef46720d10e3bf7"
server_key_iv: "93bbaf7d096859a6b0ced446e89c3f6d"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.572609 32216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 04:45:44.575256 32238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.576678 32216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 04:45:44.576815 32216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "de3aeaed68624f448b39563148adfcbe"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "4c985d522278828e7ef46720d10e3bf7"
server_key_iv: "93bbaf7d096859a6b0ced446e89c3f6d"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.576924 32216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:44.594187 32216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:44.597079 32216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:44.597285 32216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:44.597882 32216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:44.598706 32216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:44.598775 32216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.598850 32216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:44.598892 32216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.608054 32216 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:41819
I20260808 04:45:44.608085 32404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:41819 every 8 connection(s)
I20260808 04:45:44.609047 32216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:44.609753 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 32216
I20260808 04:45:44.609908 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:44.610278 22538 external_mini_cluster.cc:1428] Setting key 66b277780852a8a454de4d0afb2411dd
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:44.622896 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.610960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:50233 (local address 127.22.2.190:38987)
0808 04:45:44.611118 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.611123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.612176 (+ 1053us) server_negotiation.cc:407] Connection header received
0808 04:45:44.613683 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.613687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.613743 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.613829 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.614991 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.615528 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.616693 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.616933 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.617047 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.619465 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.619469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.619472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.619507 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.620792 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.621300 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.621304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.621305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.621393 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.621735 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.621739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.621741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.621935 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.622077 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.622591 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.622716 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 04:45:44.623870 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.611508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:50233)
0808 04:45:44.612034 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.612078 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.613472 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.613997 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.614004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.614362 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.614843 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.614857 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.615657 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.615661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.616566 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.616576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.617088 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.617092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.617262 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.617843 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.617859 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.619291 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.620913 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.620919 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.620934 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.621189 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.621491 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.621494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.621496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.621634 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.622075 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.622082 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.622337 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.622548 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.622855 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":122,"threads_started":1}
I20260808 04:45:44.625026 32405 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38987
I20260808 04:45:44.625272 32405 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:44.625767 32405 heartbeater.cc:511] Master 127.22.2.190:38987 requested a full tablet report, sending...
I20260808 04:45:44.626850 31716 ts_manager.cc:194] Registered new tserver with Master: de3aeaed68624f448b39563148adfcbe (127.22.2.131:41819)
I20260808 04:45:44.627461 31716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:50233
I20260808 04:45:44.636482 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.2.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.189.keytab.
Entry for principal oryx/127.22.2.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.189.keytab.
I20260808 04:45:44.667409 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:39161
--webserver_interface=127.22.2.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.189.keytab
--principal=oryx/127.22.2.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38987,127.22.2.189:39161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:44.791618 32410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:44.791885 32410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:44.791925 32410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:44.796017 32410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:44.796083 32410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:44.796104 32410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:44.796150 32410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:44.796183 32410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:44.801525 32410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38987,127.22.2.189:39161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.189.keytab
--principal=oryx/127.22.2.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:39161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.2.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
W20260808 04:45:44.801760 32410 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.2.190:38987,127.22.2.189:39161'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 04:45:44.802675 32410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:44.803730 32410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:44.812961 32419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:44.812949 32424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:44.812949 32421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:44.813400 32410 server_base.cc:1034] running on GCE node
I20260808 04:45:44.813954 32410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:44.815089 32410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:44.816269 32410 hybrid_clock.cc:648] HybridClock initialized: now 1786164344816232 us; error 41 us; skew 500 ppm
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:44.819248 32410 init.cc:377] Logged in from keytab as oryx/127.22.2.189@KRBTEST.COM (short username oryx)
I20260808 04:45:44.820221 32410 webserver.cc:460] Webserver started at http://127.22.2.189:38703/ using document root <none> and password file <none>
I20260808 04:45:44.820881 32410 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:44.820977 32410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:44.821231 32410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:44.823303 32410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data/instance:
uuid: "58f82dfd6d4b4e4f93adc6e228615efc"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "7079463c67a77bb87a54fc854ca73ff0"
server_key_iv: "c78ac4a37894416f7b76a0a3235f76cc"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.823948 32410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal/instance:
uuid: "58f82dfd6d4b4e4f93adc6e228615efc"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "7079463c67a77bb87a54fc854ca73ff0"
server_key_iv: "c78ac4a37894416f7b76a0a3235f76cc"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.827879 32410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 04:45:44.830631 32435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.831904 32410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:44.832154 32410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
uuid: "58f82dfd6d4b4e4f93adc6e228615efc"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "7079463c67a77bb87a54fc854ca73ff0"
server_key_iv: "c78ac4a37894416f7b76a0a3235f76cc"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.832278 32410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:44.846243 32410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:44.849495 32410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:44.849694 32410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:44.857072 32410 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:39161
I20260808 04:45:44.857434 32503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:39161 every 8 connection(s)
I20260808 04:45:44.858172 32410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
I20260808 04:45:44.861137 32504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:44.864456 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 32410
I20260808 04:45:44.865288 32504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.189@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 04:45:44.878716 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.867326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.22.2.190:38987)
0808 04:45:44.867490 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.867495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.867834 (+ 339us) server_negotiation.cc:407] Connection header received
0808 04:45:44.868226 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.868229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.868285 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.868392 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.869466 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.870210 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.871406 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.871673 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.871810 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.874667 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.874671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.874673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.874696 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.876435 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.877055 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.877058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.877059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.877185 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.877537 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.877540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.877541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.877792 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.877925 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.878424 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.878540 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 04:45:44.879515 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.866268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.22.2.189:39161)
0808 04:45:44.866740 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.866758 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.866795 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:45:44.867636 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.867661 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.868016 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.868287 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.869122 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.870217 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.871196 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.871671 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.871843 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.875180 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.875186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.875197 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.875219 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.877151 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.877874 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.877880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.877883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.877961 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.878252 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.878255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.878257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.878480 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.878692 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.879226 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.879346 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":162,"threads_started":1}
I20260808 04:45:44.879519 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.867249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:48602)
0808 04:45:44.867661 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.867714 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.867999 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.868483 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.868489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.868942 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.869313 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.869328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.870402 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.870405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.871277 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.871286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.871855 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.871859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.871977 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.872741 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.872759 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.874496 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.876582 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.876590 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.876601 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.876942 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.877287 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.877290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.877292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.877444 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.877926 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.877934 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.878177 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.878364 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.878675 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":125,"threads_started":1}
I20260808 04:45:44.881219 32504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164344, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.189@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 04:45:44.893428 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.885050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.0.0.1:46284)
0808 04:45:44.885188 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.885203 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.885283 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.885510 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.885513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.885705 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.885821 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.885828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.886522 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.886526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.887359 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.887370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.887805 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.887809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.887928 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.888409 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.888424 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.889784 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.891818 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.891821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.891823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.892050 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.892386 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.892388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.892390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.892476 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.893043 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.893046 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.893123 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.893212 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.893315 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 04:45:44.893575 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.885159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.22.2.189:39161)
0808 04:45:44.885269 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.885272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.885283 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:45:44.885371 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.885373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.885407 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.885471 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.885935 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.886419 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.887477 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.887701 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.887779 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.889924 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.889929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.889932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.889958 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.891556 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.892190 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.892193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.892196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.892281 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.892735 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.892738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.892740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.892943 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.893042 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.893286 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.893396 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 04:45:44.896797 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Bootstrap starting.
I20260808 04:45:44.899041 32504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:44.899708 32504 log.cc:824] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:44.901856 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: No bootstrap required, opened a new log
I20260808 04:45:44.905308 32504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } }
I20260808 04:45:44.905519 32504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:44.905555 32504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f82dfd6d4b4e4f93adc6e228615efc, State: Initialized, Role: FOLLOWER
I20260808 04:45:44.905969 32504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } }
I20260808 04:45:44.906445 32516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } } }
I20260808 04:45:44.906621 32516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:45:44.907389 32504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: configured and running, proceeding with master startup.
W20260808 04:45:44.912269 32531 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:45:44.912387 32531 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:45:44.914250 32410 master_runner.cc:428] Detected that this master 58f82dfd6d4b4e4f93adc6e228615efc is joining an existing cluster
I20260808 04:45:44.914312 32410 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.2.189:39161
I20260808 04:45:44.925581 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.916955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.22.2.189:39161)
0808 04:45:44.917084 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.917088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.917232 (+ 144us) server_negotiation.cc:407] Connection header received
0808 04:45:44.917344 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.917349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.917387 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.917485 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.918017 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.918908 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.920129 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.920476 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.920596 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.921936 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.921940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.921942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.921967 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.923810 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.924293 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.924298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.924304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.924406 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.924757 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.924760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.924763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.924960 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.925031 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.925311 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.925416 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":50}
I20260808 04:45:44.925999 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.916810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.0.0.1:46286)
0808 04:45:44.917114 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.917129 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.917268 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.917503 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.917506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.917786 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.917899 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.917905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.919052 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.919056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.920008 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.920020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.920716 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.920719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.920787 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.921400 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.921423 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.921779 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.923947 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.923950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.923952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.924185 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.924519 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.924548 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.924550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.924646 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.925089 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.925091 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.925163 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.925716 (+ 553us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.925837 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":98,"threads_started":1}
I20260808 04:45:44.926168 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.916432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48610 (local address 127.22.2.190:38987)
0808 04:45:44.916606 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.916613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.917323 (+ 710us) server_negotiation.cc:407] Connection header received
0808 04:45:44.917374 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.917377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.917426 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.917521 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.918225 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.919087 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.919990 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.920369 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.920482 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.922320 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.922326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.922333 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.922370 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.924188 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.924748 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.924752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.924754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.924849 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.925223 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.925227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.925229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.925421 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.925537 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.925872 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.925992 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 04:45:44.926654 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.916370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:48610)
0808 04:45:44.916735 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.916768 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.917072 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.917649 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.917653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.917916 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.918087 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.918097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.919206 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.919212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.919850 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.919858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.920508 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.920514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.920645 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.921714 (+ 1069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.921730 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.922191 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.924332 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.924335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.924339 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.924641 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.924958 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.924961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.924964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.925102 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.925528 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.925531 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.925634 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.926317 (+ 683us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.926465 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
Aug 08 04:45:44 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164343, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 04:45:44.941570 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.931295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.22.2.189:39161)
0808 04:45:44.931421 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.931424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.932179 (+ 755us) server_negotiation.cc:407] Connection header received
0808 04:45:44.932421 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.932427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.932491 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.932611 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.933429 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.934185 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.935552 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.935788 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.935849 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.938164 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.938167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.938170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.938200 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.939718 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.940210 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.940213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.940215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.940296 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.940627 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.940631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.940633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.940850 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.940945 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.941294 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.941416 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:45:44.941596 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.931226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.0.0.1:46296)
0808 04:45:44.932037 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.932089 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.932225 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.932652 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.932659 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.933151 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.933289 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.933299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.934319 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.934325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.935396 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.935411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.935972 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.935974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.936050 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.936604 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.936623 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.938028 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.939848 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.939857 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.939864 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.940098 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.940400 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.940404 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.940406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.940497 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.940961 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.940969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.941090 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.941240 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.941434 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":99,"threads_started":1}
I20260808 04:45:44.943184 31716 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.2.189:39161: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc"
member_type: NON_VOTER
last_known_addr {
host: "127.22.2.189"
port: 39161
}
attrs {
promote: true
}
}
dest_uuid: "d2416c14ab864738b27a3a0891a4b474"
cas_config_opid_index: -1
WARNING: no policy specified for oryx/127.22.2.188@KRBTEST.COM; defaulting to no policy
I20260808 04:45:44.943933 31716 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } }
I20260808 04:45:44.945881 32546 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 58f82dfd6d4b4e4f93adc6e228615efc (127.22.2.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.188@KRBTEST.COM" created.
I20260808 04:45:44.946760 32472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:44.947767 32546 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.2.189:39161
I20260808 04:45:44.948221 32410 master.cc:537] Master@127.22.2.189:39161 shutting down...
I20260808 04:45:44.948248 32472 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Refusing update from remote peer d2416c14ab864738b27a3a0891a4b474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 04:45:44.948894 32545 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } }
I20260808 04:45:44.949015 32545 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:44.949432 32546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 04:45:44.952612 31702 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260808 04:45:44.953747 32410 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:44.954033 32410 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:44.954162 32410 tablet_replica.cc:330] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: stopping tablet replica
W20260808 04:45:44.956869 31702 consensus_peers.cc:487] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 -> Peer 58f82dfd6d4b4e4f93adc6e228615efc (127.22.2.189:39161): Couldn't send request to peer 58f82dfd6d4b4e4f93adc6e228615efc. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.188.keytab.
Entry for principal oryx/127.22.2.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.188.keytab.
I20260808 04:45:44.963982 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:35545
--webserver_interface=127.22.2.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.188.keytab
--principal=oryx/127.22.2.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38987,127.22.2.189:39161,127.22.2.188:35545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:45:44.969019 32410 master.cc:552] Master@127.22.2.189:39161 shutdown complete.
I20260808 04:45:44.969075 32410 master_runner.cc:305] Clearing existing system tablet
I20260808 04:45:44.969328 32410 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:44.969388 32410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:44.970764 32410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:44.971698 32554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:44.971972 32410 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:44.972100 32410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:44.972133 32410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
uuid: "58f82dfd6d4b4e4f93adc6e228615efc"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "7079463c67a77bb87a54fc854ca73ff0"
server_key_iv: "c78ac4a37894416f7b76a0a3235f76cc"
server_key_version: "encryptionkey@0"
I20260808 04:45:44.974355 32410 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:44.976727 32410 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 04:45:44.976817 32410 log.cc:1196] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 04:45:44.977097 32410 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Deleting consensus metadata
I20260808 04:45:44.977273 32410 master_runner.cc:315] Copying system tablet from 127.22.2.190:38987
I20260808 04:45:44.978578 32410 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: tablet copy: Beginning tablet copy session from remote peer at address 127.22.2.190:38987
I20260808 04:45:44.988579 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.979370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.22.2.190:38987)
0808 04:45:44.979497 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:44.979502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:44.980113 (+ 611us) server_negotiation.cc:407] Connection header received
0808 04:45:44.980255 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:44.980258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:44.980317 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:44.980407 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:44.981047 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.982009 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.983198 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.983549 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.983656 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.985010 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:44.985015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:44.985018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:44.985048 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:44.986743 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.987224 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.987227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.987229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.987320 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:44.987627 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:44.987632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:44.987635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:44.987895 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:44.988084 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:44.988283 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 04:45:44.988400 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:45:44.988608 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:44.979256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:48620)
0808 04:45:44.980003 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:44.980019 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:44.980147 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:44.980448 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:44.980452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:44.980719 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:44.980860 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.980868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.982133 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.982137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.983070 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:44.983081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:44.983672 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:44.983676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:44.983783 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:44.984369 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:44.984384 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:44.984858 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:44.986861 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.986864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.986867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.987119 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.987422 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:44.987425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:44.987427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:44.987521 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:44.988005 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:44.988008 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:44.988107 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 04:45:44.988215 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 04:45:44.988354 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":87,"threads_started":1}
I20260808 04:45:44.989670 31752 tablet_copy_service.cc:140] P d2416c14ab864738b27a3a0891a4b474: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 58f82dfd6d4b4e4f93adc6e228615efc ({username='oryx', principal='oryx/127.22.2.189@KRBTEST.COM'} at 127.0.0.1:48620)
I20260808 04:45:44.989868 31752 tablet_copy_service.cc:161] P d2416c14ab864738b27a3a0891a4b474: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 58f82dfd6d4b4e4f93adc6e228615efc at {username='oryx', principal='oryx/127.22.2.189@KRBTEST.COM'} at 127.0.0.1:48620: session id = 58f82dfd6d4b4e4f93adc6e228615efc-00000000000000000000000000000000
I20260808 04:45:44.991756 31752 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 04:45:44.993726 32410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 04:45:44.996191 32410 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: tablet copy: Starting download of 0 data blocks...
I20260808 04:45:44.996380 32410 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: tablet copy: Starting download of 1 WAL segments...
I20260808 04:45:44.998296 32410 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 04:45:45.000061 31752 tablet_copy_service.cc:342] P d2416c14ab864738b27a3a0891a4b474: Request end of tablet copy session 58f82dfd6d4b4e4f93adc6e228615efc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.2.189@KRBTEST.COM'} at 127.0.0.1:48620
I20260808 04:45:45.000236 31752 tablet_copy_service.cc:433] P d2416c14ab864738b27a3a0891a4b474: ending tablet copy session 58f82dfd6d4b4e4f93adc6e228615efc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 58f82dfd6d4b4e4f93adc6e228615efc
I20260808 04:45:45.007972 32410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:45.009747 32563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.013509 32564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.013698 32567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:45.013862 32410 server_base.cc:1034] running on GCE node
I20260808 04:45:45.014133 32410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:45.014452 32410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:45.015602 32410 hybrid_clock.cc:648] HybridClock initialized: now 1786164345015578 us; error 42 us; skew 500 ppm
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:45.017292 32410 init.cc:377] Logged in from keytab as oryx/127.22.2.189@KRBTEST.COM (short username oryx)
I20260808 04:45:45.017747 32410 webserver.cc:460] Webserver started at http://127.22.2.189:36349/ using document root <none> and password file <none>
I20260808 04:45:45.017932 32410 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:45.017973 32410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:45.019274 32410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:45.020309 32583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.020674 32410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:45.020763 32410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
uuid: "58f82dfd6d4b4e4f93adc6e228615efc"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "7079463c67a77bb87a54fc854ca73ff0"
server_key_iv: "c78ac4a37894416f7b76a0a3235f76cc"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.020852 32410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:45.027771 32410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:45.029999 32410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:45.034932 32410 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:39161
I20260808 04:45:45.035219 32647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:39161 every 8 connection(s)
I20260808 04:45:45.035567 32410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
I20260808 04:45:45.037442 32648 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:45:45.038524 32648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Bootstrap starting.
I20260808 04:45:45.049350 32648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:45:45.049669 32648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: Bootstrap complete.
I20260808 04:45:45.050256 32648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } }
I20260808 04:45:45.050400 32648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58f82dfd6d4b4e4f93adc6e228615efc, State: Initialized, Role: LEARNER
I20260808 04:45:45.050549 32648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } }
I20260808 04:45:45.050913 32653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } }
I20260808 04:45:45.051040 32648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:45.051113 32653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: This master's current role is: LEARNER
I20260808 04:45:45.054419 32670 catalog_manager.cc:1259] Loaded cluster ID: e84031d74e684823b9b396333083bff1
I20260808 04:45:45.054484 32670 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: loading cluster ID for follower catalog manager: success
I20260808 04:45:45.056638 32670 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: acquiring CA information for follower catalog manager: success
I20260808 04:45:45.058132 32670 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 04:45:45.098752 32552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:45.098989 32552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:45.099028 32552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:45.103212 32552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:45.103271 32552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:45.103291 32552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:45.103336 32552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:45.103367 32552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:45.108860 32552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38987,127.22.2.189:39161,127.22.2.188:35545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.188.keytab
--principal=oryx/127.22.2.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:35545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.2.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:45.109946 32552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:45.110994 32552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:45.119711 32680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.119686 32684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.119681 32682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:45.120074 32552 server_base.cc:1034] running on GCE node
I20260808 04:45:45.120733 32552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:45.121836 32552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:45.123071 32552 hybrid_clock.cc:648] HybridClock initialized: now 1786164345123051 us; error 30 us; skew 500 ppm
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:45.126210 32552 init.cc:377] Logged in from keytab as oryx/127.22.2.188@KRBTEST.COM (short username oryx)
I20260808 04:45:45.127146 32552 webserver.cc:460] Webserver started at http://127.22.2.188:38129/ using document root <none> and password file <none>
I20260808 04:45:45.127673 32552 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:45.127731 32552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:45.127918 32552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:45.129997 32552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data/instance:
uuid: "eaec38a20b3845e0ad056adafb8f6afe"
format_stamp: "Formatted at 2026-08-08 04:45:45 on dist-test-slave-04bb"
server_key: "48f860ca1f18708ed179c63a2796a142"
server_key_iv: "c4973c12883c1b1bc9871da0f80613e3"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.130528 32552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal/instance:
uuid: "eaec38a20b3845e0ad056adafb8f6afe"
format_stamp: "Formatted at 2026-08-08 04:45:45 on dist-test-slave-04bb"
server_key: "48f860ca1f18708ed179c63a2796a142"
server_key_iv: "c4973c12883c1b1bc9871da0f80613e3"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.134567 32552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:45:45.137079 32694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.138362 32552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:45.138497 32552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
uuid: "eaec38a20b3845e0ad056adafb8f6afe"
format_stamp: "Formatted at 2026-08-08 04:45:45 on dist-test-slave-04bb"
server_key: "48f860ca1f18708ed179c63a2796a142"
server_key_iv: "c4973c12883c1b1bc9871da0f80613e3"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.138605 32552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:45.177095 32024 heartbeater.cc:503] Master 127.22.2.190:38987 was elected leader, sending a full tablet report...
I20260808 04:45:45.179566 32552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:45.182904 32552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:45.183104 32552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:45.192466 32552 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:35545
I20260808 04:45:45.192780 32763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:35545 every 8 connection(s)
I20260808 04:45:45.193519 32552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
I20260808 04:45:45.196393 32765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:45.200166 32765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:45:45.201346 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 32552
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 04:45:45.214524 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.202291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48632 (local address 127.22.2.190:38987)
0808 04:45:45.203106 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.203111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.203333 (+ 222us) server_negotiation.cc:407] Connection header received
0808 04:45:45.204153 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.204156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.204203 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.204277 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.205465 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.205984 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.207236 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.207504 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.207590 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.210446 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.210450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.210452 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.210475 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.212024 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.212612 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.212615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.212616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.212698 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.213086 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.213090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.213093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.213329 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.213440 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.214029 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.214159 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720}
I20260808 04:45:45.215310 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.202566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.22.2.188:35545)
0808 04:45:45.203034 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.203053 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.203094 (+ 41us) server_negotiation.cc:407] Connection header received
0808 04:45:45.203928 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.203953 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.204194 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.204597 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.205324 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.206374 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.207317 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.207770 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.207884 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.210845 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.210855 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.210871 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.210900 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.212792 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.213242 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.213247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.213248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.213318 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.213596 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.213599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.213601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.213864 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.214053 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.214272 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.214393 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":319,"spinlock_wait_cycles":32000,"thread_start_us":177,"threads_started":1}
I20260808 04:45:45.215324 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.202566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:48632)
0808 04:45:45.203085 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.203134 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.203928 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.204405 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.204423 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.204914 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.205308 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.205324 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.206118 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.206122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.207115 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.207125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.207650 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.207664 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.207847 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.208502 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.208556 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.210284 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.212134 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.212141 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.212151 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.212471 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.212792 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.212796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.212798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.212977 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.213455 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.213460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.213740 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.213934 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.214276 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":21,"thread_start_us":139,"threads_started":1}
I20260808 04:45:45.216651 32765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 04:45:45.224993 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.217140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.0.0.1:46302)
0808 04:45:45.217253 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.217269 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.217348 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.218149 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.218152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.218283 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.218374 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.218380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.219244 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.219247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.219783 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.219790 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.220189 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.220192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.220260 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.220731 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.220744 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.221888 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.223548 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.223550 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.223551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.223832 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.224079 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.224081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.224082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.224156 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.224556 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.224561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.224643 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.224728 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.224821 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 04:45:45.225065 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.217284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.22.2.189:39161)
0808 04:45:45.217754 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.217759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.217775 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:45:45.217819 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.217822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.217872 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.218119 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.218485 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.219150 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.219882 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.220096 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.220174 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.222015 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.222018 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.222020 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.222043 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.223447 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.223917 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.223919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.223921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.224000 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.224269 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.224271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.224272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.224442 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.224565 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.224787 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.224946 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":188,"threads_started":1}
I20260808 04:45:45.225872 32765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 04:45:45.232991 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.226362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35545 (local address 127.0.0.1:38354)
0808 04:45:45.226475 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.226488 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.226561 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.226748 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.226750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.226873 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.226938 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.226942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.227618 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.227621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.228100 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.228107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.228515 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.228517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.228655 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.229021 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.229032 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.230156 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.231586 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.231588 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.231590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.231774 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.232046 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.232049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.232050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.232134 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.232500 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.232502 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.232634 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.232724 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.232817 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 04:45:45.232996 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.226420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.22.2.188:35545)
0808 04:45:45.226533 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.226536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.226547 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:45:45.226627 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.226629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.226661 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.226722 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.227052 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.227523 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.228201 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.228421 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.228501 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.230321 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.230324 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.230326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.230347 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.231487 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.231860 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.231862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.231863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.231933 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.232254 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.232256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.232257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.232414 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.232487 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.232785 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.232872 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 04:45:45.236611 32765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Bootstrap starting.
I20260808 04:45:45.238929 32765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:45.239671 32765 log.cc:824] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:45.241760 32765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: No bootstrap required, opened a new log
I20260808 04:45:45.244936 32765 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } }
I20260808 04:45:45.245141 32765 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:45.245175 32765 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaec38a20b3845e0ad056adafb8f6afe, State: Initialized, Role: FOLLOWER
I20260808 04:45:45.245621 32765 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } }
I20260808 04:45:45.246292 307 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } } }
I20260808 04:45:45.246430 307 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:45:45.246959 32765 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: configured and running, proceeding with master startup.
W20260808 04:45:45.252112 328 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:45:45.252194 328 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:45:45.255533 32552 master_runner.cc:428] Detected that this master eaec38a20b3845e0ad056adafb8f6afe is joining an existing cluster
I20260808 04:45:45.255601 32552 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.2.188:35545
I20260808 04:45:45.269320 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 31818
I20260808 04:45:45.287082 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.258612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.22.2.189:39161)
0808 04:45:45.258738 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.258742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.259238 (+ 496us) server_negotiation.cc:407] Connection header received
0808 04:45:45.259424 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.259427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.259474 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.259522 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.260107 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.260942 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.263046 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.263700 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.263784 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.283240 (+ 19456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.283246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.283248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.283278 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.285089 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.285669 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.285673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.285675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.285775 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.286076 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.286079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.286081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.286314 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.286387 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.286770 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.286908 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 04:45:45.287350 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.257920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48636 (local address 127.22.2.190:38987)
0808 04:45:45.258037 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.258042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.258549 (+ 507us) server_negotiation.cc:407] Connection header received
0808 04:45:45.258741 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.258744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.258800 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.258867 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.260408 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.261199 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.262372 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.262743 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.262850 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.282653 (+ 19803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.282660 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.282666 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.282709 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.285104 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.285774 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.285782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.285786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.285884 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.286213 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.286219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.286223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.286493 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.286572 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.286956 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.287100 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 04:45:45.287659 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.258831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.0.0.1:46306)
0808 04:45:45.259152 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.259168 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.259297 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.259653 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.259656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.259859 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.259973 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.259979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.261079 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.261082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.262901 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.262913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.263964 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.263967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.264059 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.264473 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.264488 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.283069 (+ 18581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.285229 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.285234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.285239 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.285548 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.285888 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.285891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.285893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.285977 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.286463 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.286466 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.286576 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.287293 (+ 717us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.287460 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":4,"thread_start_us":82,"threads_started":1}
I20260808 04:45:45.288753 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.258167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:48636)
0808 04:45:45.258415 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.258430 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.258622 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.259054 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.259057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.259278 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.260288 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.260293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.261334 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.261336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.262217 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.262235 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.262976 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.262979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.263082 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.263533 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.263543 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.282431 (+ 18888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.285255 (+ 2824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.285262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.285267 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.285642 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.286006 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.286012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.286016 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.286109 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.286650 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.286656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.286764 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.288361 (+ 1597us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.288559 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":45,"thread_start_us":101,"threads_started":1}
I20260808 04:45:45.289081 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:45035
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:38987
--webserver_port=35553
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161,127.22.2.188:35545 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:45:45.292631 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.258000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.22.2.188:35545)
0808 04:45:45.258107 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.258111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.258410 (+ 299us) server_negotiation.cc:407] Connection header received
0808 04:45:45.258568 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.258571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.258607 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.258652 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.259337 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.260115 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.261328 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.261667 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.261744 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.263244 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.263250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.263256 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.263278 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.282699 (+ 19421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.291229 (+ 8530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.291234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.291235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.291319 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.291608 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.291612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.291614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.291826 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.291926 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.292263 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.292433 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":42}
I20260808 04:45:45.292945 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.257917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35545 (local address 127.0.0.1:38356)
0808 04:45:45.258214 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.258229 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.258474 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.258787 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.258793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.259065 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.259216 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.259224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.260219 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.260225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.261185 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.261197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.261895 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.261898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.261982 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.262545 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.262559 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.263091 (+ 532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.290822 (+ 27731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.290827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.290830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.291108 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.291426 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.291429 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.291434 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.291524 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.291945 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.291950 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.292088 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.292211 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.292781 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":120,"threads_started":1}
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164343, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 04:45:45.304569 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.293611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35545 (local address 127.0.0.1:38358)
0808 04:45:45.293878 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.293895 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.293977 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.294774 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.294777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.294988 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.295124 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.295136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.296270 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.296276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.297295 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.297321 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.298136 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.298142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.298266 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.298794 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.298812 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.300684 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.302826 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.302832 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.302836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.303109 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.303479 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.303485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.303489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.303590 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.304054 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.304057 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.304148 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.304241 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.304386 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260808 04:45:45.305016 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.294121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38358 (local address 127.22.2.188:35545)
0808 04:45:45.294347 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.294353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.294370 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:45:45.294451 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.294457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.294537 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.294622 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.295276 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.296120 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.297469 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.297875 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.297995 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.300850 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.300857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.300863 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.300893 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.302683 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.303241 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.303247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.303252 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.303356 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.303713 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.303718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.303722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.303918 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.304007 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.304675 (+ 668us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.304794 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260808 04:45:45.305900 31716 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.2.188:35545: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe"
member_type: NON_VOTER
last_known_addr {
host: "127.22.2.188"
port: 35545
}
attrs {
promote: true
}
}
dest_uuid: "d2416c14ab864738b27a3a0891a4b474"
cas_config_opid_index: 5
I20260808 04:45:45.306541 31716 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } }
I20260808 04:45:45.307871 32545 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.309455 32545 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.2.188:35545
I20260808 04:45:45.309619 32545 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.309708 32545 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:45.310564 32552 master.cc:537] Master@127.22.2.188:35545 shutting down...
I20260808 04:45:45.310698 32729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:45.312211 32729 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Refusing update from remote peer d2416c14ab864738b27a3a0891a4b474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 04:45:45.313930 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.308079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.0.0.1:46320)
0808 04:45:45.308318 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.308332 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.308400 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.309025 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.309028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.309046 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:45.309122 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.309127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.310170 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.310174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.311321 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.311332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.313415 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.313422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.313537 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.313595 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.313678 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.313761 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260808 04:45:45.314209 32546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:45.314392 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.308577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.22.2.189:39161)
0808 04:45:45.308695 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.308699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.308711 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:45:45.308755 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.308758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.308814 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.308904 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:45.309234 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.310037 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.312282 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.313040 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.313170 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.313343 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.314182 (+ 839us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.314266 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:45.315212 32613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 LEARNER]: Refusing update from remote peer d2416c14ab864738b27a3a0891a4b474: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 04:45:45.315671 32546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260808 04:45:45.316604 31699 consensus_peers.cc:487] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 -> Peer eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188:35545): Couldn't send request to peer eaec38a20b3845e0ad056adafb8f6afe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 04:45:45.318046 32613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.320158 32653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2416c14ab864738b27a3a0891a4b474. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.320328 32653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: This master's current role is: LEARNER
I20260808 04:45:45.320446 32653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: true } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.320533 32653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: This master's current role is: LEARNER
I20260808 04:45:45.322552 32546 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: attempting to promote NON_VOTER 58f82dfd6d4b4e4f93adc6e228615efc to VOTER
I20260808 04:45:45.323051 32552 raft_consensus.cc:2229] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:45.323041 32546 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } }
I20260808 04:45:45.323338 32552 raft_consensus.cc:2258] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:45.323450 32552 tablet_replica.cc:330] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: stopping tablet replica
I20260808 04:45:45.324434 32613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 LEARNER]: Refusing update from remote peer d2416c14ab864738b27a3a0891a4b474: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 04:45:45.325237 32545 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 04:45:45.325810 31699 consensus_peers.cc:487] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 -> Peer eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188:35545): Couldn't send request to peer eaec38a20b3845e0ad056adafb8f6afe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 04:45:45.326882 339 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 58f82dfd6d4b4e4f93adc6e228615efc (127.22.2.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.328840 32613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 58f82dfd6d4b4e4f93adc6e228615efc (127.22.2.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.328923 32545 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.329025 32545 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:45.329202 32545 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.329294 32545 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:45.330390 341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.330511 341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:45:45.339799 32552 master.cc:552] Master@127.22.2.188:35545 shutdown complete.
I20260808 04:45:45.339862 32552 master_runner.cc:305] Clearing existing system tablet
I20260808 04:45:45.340113 32552 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:45.340171 32552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:45.341607 32552 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:45.342552 343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.342814 32552 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:45.342941 32552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:45.342978 32552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
uuid: "eaec38a20b3845e0ad056adafb8f6afe"
format_stamp: "Formatted at 2026-08-08 04:45:45 on dist-test-slave-04bb"
server_key: "48f860ca1f18708ed179c63a2796a142"
server_key_iv: "c4973c12883c1b1bc9871da0f80613e3"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.345202 32552 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:45.347630 32552 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 04:45:45.347755 32552 log.cc:1196] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 04:45:45.348029 32552 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Deleting consensus metadata
I20260808 04:45:45.348210 32552 master_runner.cc:315] Copying system tablet from 127.22.2.190:38987
I20260808 04:45:45.349476 32552 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.2.190:38987
I20260808 04:45:45.359277 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.350016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:37562)
0808 04:45:45.350454 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.350480 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.350633 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.351012 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.351015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.351235 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.351651 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.351661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.352881 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.352887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.353740 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.353751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.354329 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.354334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.354421 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.355017 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.355031 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.355455 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.357602 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.357606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.357608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.357860 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.358185 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.358188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.358190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.358293 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.358729 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.358732 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.358833 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.358942 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.359078 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":85,"threads_started":1}
I20260808 04:45:45.359288 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.350130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37562 (local address 127.22.2.190:38987)
0808 04:45:45.350271 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.350276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.350596 (+ 320us) server_negotiation.cc:407] Connection header received
0808 04:45:45.350735 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.350739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.350797 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.350883 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.351824 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.352720 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.353873 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.354196 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.354325 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.355687 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.355692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.355694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.355722 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.357503 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.357974 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.357978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.357980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.358079 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.358399 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.358403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.358405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.358623 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.358724 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.359021 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.359137 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:45.359987 31752 tablet_copy_service.cc:140] P d2416c14ab864738b27a3a0891a4b474: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer eaec38a20b3845e0ad056adafb8f6afe ({username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:37562)
I20260808 04:45:45.360088 31752 tablet_copy_service.cc:161] P d2416c14ab864738b27a3a0891a4b474: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer eaec38a20b3845e0ad056adafb8f6afe at {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:37562: session id = eaec38a20b3845e0ad056adafb8f6afe-00000000000000000000000000000000
I20260808 04:45:45.361236 31752 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 04:45:45.362474 32552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 04:45:45.364804 32552 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: tablet copy: Starting download of 0 data blocks...
I20260808 04:45:45.364980 32552 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: tablet copy: Starting download of 1 WAL segments...
I20260808 04:45:45.366498 32552 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 04:45:45.368295 31752 tablet_copy_service.cc:342] P d2416c14ab864738b27a3a0891a4b474: Request end of tablet copy session eaec38a20b3845e0ad056adafb8f6afe-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:37562
I20260808 04:45:45.368373 31752 tablet_copy_service.cc:433] P d2416c14ab864738b27a3a0891a4b474: ending tablet copy session eaec38a20b3845e0ad056adafb8f6afe-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer eaec38a20b3845e0ad056adafb8f6afe
I20260808 04:45:45.376590 32552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:45.378376 354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.378684 358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.378651 356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:45.379189 32552 server_base.cc:1034] running on GCE node
I20260808 04:45:45.379386 32552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:45.379693 32552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:45.380858 32552 hybrid_clock.cc:648] HybridClock initialized: now 1786164345380842 us; error 30 us; skew 500 ppm
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:45.382606 32552 init.cc:377] Logged in from keytab as oryx/127.22.2.188@KRBTEST.COM (short username oryx)
I20260808 04:45:45.383097 32552 webserver.cc:460] Webserver started at http://127.22.2.188:40235/ using document root <none> and password file <none>
I20260808 04:45:45.383296 32552 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:45.383358 32552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:45.384833 32552 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:45.385912 368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.386284 32552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:45.386372 32552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
uuid: "eaec38a20b3845e0ad056adafb8f6afe"
format_stamp: "Formatted at 2026-08-08 04:45:45 on dist-test-slave-04bb"
server_key: "48f860ca1f18708ed179c63a2796a142"
server_key_iv: "c4973c12883c1b1bc9871da0f80613e3"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.386456 32552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:45.404377 32552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:45.406590 32552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:45.411684 32552 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:35545
I20260808 04:45:45.411774 442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:35545 every 8 connection(s)
I20260808 04:45:45.412246 32552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
I20260808 04:45:45.413879 32211 heartbeater.cc:503] Master 127.22.2.190:38987 was elected leader, sending a full tablet report...
I20260808 04:45:45.414700 443 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:45:45.416292 443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Bootstrap starting.
I20260808 04:45:45.429322 443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:45:45.429677 443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Bootstrap complete.
I20260808 04:45:45.430413 443 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } }
I20260808 04:45:45.430583 443 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eaec38a20b3845e0ad056adafb8f6afe, State: Initialized, Role: LEARNER
I20260808 04:45:45.430733 443 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } }
I20260808 04:45:45.431095 446 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.431247 443 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:45.431274 446 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: This master's current role is: LEARNER
I20260808 04:45:45.435057 462 catalog_manager.cc:1259] Loaded cluster ID: e84031d74e684823b9b396333083bff1
I20260808 04:45:45.435110 462 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: loading cluster ID for follower catalog manager: success
I20260808 04:45:45.436847 462 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: acquiring CA information for follower catalog manager: success
I20260808 04:45:45.437948 462 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 04:45:45.444741 338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:45.444998 338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:45.445103 338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:45.449136 338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:45.449208 338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:45.449316 338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:45.454727 338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:45035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=35553
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161,127.22.2.188:35545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:45.455766 338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:45.456909 338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:45.466838 468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.466838 470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.466879 473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:45.467175 338 server_base.cc:1034] running on GCE node
I20260808 04:45:45.467630 338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:45.468250 338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:45.469422 338 hybrid_clock.cc:648] HybridClock initialized: now 1786164345469383 us; error 49 us; skew 500 ppm
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:45.472164 338 init.cc:377] Logged in from keytab as oryx/127.22.2.129@KRBTEST.COM (short username oryx)
I20260808 04:45:45.473080 338 webserver.cc:460] Webserver started at http://127.22.2.129:35553/ using document root <none> and password file <none>
I20260808 04:45:45.473636 338 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:45.473735 338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:45.477531 338 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:45.479529 484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.480867 338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:45.480999 338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "b8e130596245467ab759328b94fd2ed2"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "5db7ef0866762ef7c7714968aae19fa8"
server_key_iv: "3f0b7e0e9431e8695bdee628bf6cc65e"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.481498 338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:45.496305 338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:45.499032 338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:45.499183 338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:45.499701 338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:45.500488 338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:45.500576 338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.500633 338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:45.500653 338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.510113 338 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:45035
I20260808 04:45:45.510151 648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:45035 every 8 connection(s)
I20260808 04:45:45.511101 338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:45.518003 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 338
I20260808 04:45:45.518189 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 32031
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 04:45:45.530117 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.513112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:37291 (local address 127.22.2.188:35545)
0808 04:45:45.513417 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.513420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.514745 (+ 1325us) server_negotiation.cc:407] Connection header received
0808 04:45:45.515666 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.515670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.515729 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.515926 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.517564 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.518616 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.521036 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.521584 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.522374 (+ 790us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.525393 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.525398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.525401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.525429 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.527288 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.527874 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.527878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.527880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.527966 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.528417 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.528420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.528422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.528668 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.528774 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.529685 (+ 911us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.529864 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":148,"threads_started":1}
I20260808 04:45:45.530738 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.513016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:51763 (local address 127.22.2.189:39161)
0808 04:45:45.513153 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.513160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.514190 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 04:45:45.515666 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.515670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.515732 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.515800 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.517574 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.518374 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.520316 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.520686 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.520767 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.525227 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.525232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.525235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.525266 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.527079 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.527874 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.527878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.527880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.527966 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.528417 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.528420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.528422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.528668 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.528751 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.530471 (+ 1720us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.530587 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:45:45.531697 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.514792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:43721 (local address 127.22.2.190:38987)
0808 04:45:45.514946 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.514951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.515167 (+ 216us) server_negotiation.cc:407] Connection header received
0808 04:45:45.515650 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.515653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.515730 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.515818 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.517532 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.518371 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.520317 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.520687 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.520767 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.525458 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.525466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.525473 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.526077 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.529007 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.529717 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.529724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.529729 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.529845 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.530289 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.530296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.530301 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.530635 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.530737 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.531240 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.531427 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 04:45:45.531731 657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.513459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.22.2.129:51763)
0808 04:45:45.513988 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.514030 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.515419 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.516021 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.516033 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.516782 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.517394 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.517403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.518707 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.518710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.520169 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.520181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.520976 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.520980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.521355 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.522308 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.522337 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.524999 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.527239 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.527248 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.527265 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.527736 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.528083 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.528086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.528088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.528303 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.528842 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.528849 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.529282 (+ 433us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.531013 (+ 1731us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.531145 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":423,"spinlock_wait_cycles":18560,"thread_start_us":155,"threads_started":1}
I20260808 04:45:45.531924 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.513459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35545 (local address 127.22.2.129:37291)
0808 04:45:45.514485 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.514500 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.515419 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.516078 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.516080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.516776 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.517394 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.517403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.518779 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.518786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.520818 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.520834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.522599 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.522606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.522754 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.523373 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.523396 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.525237 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.527404 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.527407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.527409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.527736 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.528083 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.528086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.528088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.528300 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.528842 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.528849 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.529291 (+ 442us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.530313 (+ 1022us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.530665 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":851,"spinlock_wait_cycles":89728,"thread_start_us":728,"threads_started":1}
I20260808 04:45:45.532487 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.514672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:43721)
0808 04:45:45.514981 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.514992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.515428 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.516019 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.516026 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.516767 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.517381 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.517397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.518544 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.518550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.520169 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.520181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.520976 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.520980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.521353 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.522348 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.522360 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.525228 (+ 2868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.529181 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.529188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.529193 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.529566 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.530000 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.530007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.530012 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.530142 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.530838 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.530844 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.530983 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.531967 (+ 984us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.532165 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":278,"spinlock_wait_cycles":57856,"thread_start_us":88,"threads_started":1}
I20260808 04:45:45.533463 653 heartbeater.cc:347] Connected to a master server at 127.22.2.189:39161
I20260808 04:45:45.533582 649 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38987
I20260808 04:45:45.533700 653 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.533703 649 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.534221 653 heartbeater.cc:511] Master 127.22.2.189:39161 requested a full tablet report, sending...
I20260808 04:45:45.534224 649 heartbeater.cc:511] Master 127.22.2.190:38987 requested a full tablet report, sending...
I20260808 04:45:45.535542 31716 ts_manager.cc:194] Re-registered known tserver with Master: b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035)
I20260808 04:45:45.536073 32593 ts_manager.cc:194] Registered new tserver with Master: b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035)
I20260808 04:45:45.536126 31716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:43721
I20260808 04:45:45.546175 650 heartbeater.cc:347] Connected to a master server at 127.22.2.188:35545
I20260808 04:45:45.546270 650 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.546536 650 heartbeater.cc:511] Master 127.22.2.188:35545 requested a full tablet report, sending...
I20260808 04:45:45.547853 391 ts_manager.cc:194] Registered new tserver with Master: b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035)
I20260808 04:45:45.552789 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:46229
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:38987
--webserver_port=44519
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161,127.22.2.188:35545 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:45:45.629635 32405 heartbeater.cc:503] Master 127.22.2.190:38987 was elected leader, sending a full tablet report...
W20260808 04:45:45.668306 660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:45.668620 660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:45.668670 660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:45.672545 660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:45.672598 660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:45.672654 660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:45.677802 660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:46229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=44519
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161,127.22.2.188:35545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:45.678761 660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:45.679745 660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:45.689963 671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.689952 668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.689952 665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:45.690395 660 server_base.cc:1034] running on GCE node
I20260808 04:45:45.690773 660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:45.691421 660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:45.692600 660 hybrid_clock.cc:648] HybridClock initialized: now 1786164345692596 us; error 40 us; skew 500 ppm
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:45.695340 660 init.cc:377] Logged in from keytab as oryx/127.22.2.130@KRBTEST.COM (short username oryx)
I20260808 04:45:45.696245 660 webserver.cc:460] Webserver started at http://127.22.2.130:44519/ using document root <none> and password file <none>
I20260808 04:45:45.696907 660 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:45.697026 660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:45.701040 660 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:45:45.703151 681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.704416 660 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:45.704576 660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "d8c1979ae5fb4ef0a97987256b08aa2c"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "69070d4bc0462035b8288c71bceae6e1"
server_key_iv: "67080a2a00af946a509b6b7b75c52e18"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.705138 660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:45.735630 660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:45.739102 660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:45.739315 660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:45.739933 660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:45.740922 660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:45.740994 660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.741067 660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:45.741117 660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.751951 660 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:46229
I20260808 04:45:45.752015 834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:46229 every 8 connection(s)
I20260808 04:45:45.753074 660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:45.760183 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 660
I20260808 04:45:45.760361 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 32216
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 04:45:45.773249 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.755051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53009 (local address 127.22.2.190:38987)
0808 04:45:45.755225 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.755230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.757202 (+ 1972us) server_negotiation.cc:407] Connection header received
0808 04:45:45.757820 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.757823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.757877 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.757935 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.760067 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.760944 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763124 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.763489 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763572 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.768127 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.768131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.768137 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.768174 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.770274 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771007 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.771014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.771019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.771116 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771649 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.771655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.771660 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.771931 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.772014 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.772830 (+ 816us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.772990 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 04:45:45.773443 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.756882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:32837 (local address 127.22.2.189:39161)
0808 04:45:45.757006 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.757011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.758610 (+ 1599us) server_negotiation.cc:407] Connection header received
0808 04:45:45.758844 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.758847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.758911 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.758968 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.760082 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.760953 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763157 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.763569 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763674 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.769049 (+ 5375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.769053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.769055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.769084 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.771177 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771744 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.771747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.771749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.771852 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.772205 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.772208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.772210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.772428 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.772488 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.773178 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.773285 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:45.773443 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.755059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:37639 (local address 127.22.2.188:35545)
0808 04:45:45.755242 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.755247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.756359 (+ 1112us) server_negotiation.cc:407] Connection header received
0808 04:45:45.757812 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.757816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.757895 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.757950 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.760067 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.760936 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763123 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.763489 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763574 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.769003 (+ 5429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.769007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.769010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.769039 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.771096 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771743 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.771747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.771749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.771848 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.772205 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.772208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.772210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.772721 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.772853 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.773146 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.773288 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 04:45:45.774192 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:41819
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:38987
--webserver_port=40503
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161,127.22.2.188:35545 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:45:45.775362 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.755493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:53009)
0808 04:45:45.756939 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.756981 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.757559 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.758356 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.758365 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.758997 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.759891 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.759912 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.761133 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.761136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.762978 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.762990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.763784 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.764211 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.765176 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.765201 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.767920 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.770428 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.770437 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.770457 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.770879 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.771263 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.771274 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.771520 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.772105 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.772114 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.772453 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.774027 (+ 1574us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.774216 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1330,"mutex_wait_us":809,"spinlock_wait_cycles":16896,"thread_start_us":156,"threads_started":1}
I20260808 04:45:45.775369 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.755493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35545 (local address 127.22.2.130:37639)
0808 04:45:45.756049 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.756140 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.757561 (+ 1421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.758356 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.758365 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.758997 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.759891 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.759912 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.761133 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.761136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.762980 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.762990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.763784 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.764214 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.765271 (+ 1057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.765285 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.768857 (+ 3572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.771236 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.771246 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.771623 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.771971 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.771976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.772098 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.772845 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.772848 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.772950 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.773652 (+ 702us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.774169 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":709,"spinlock_wait_cycles":10880,"thread_start_us":222,"threads_started":1}
I20260808 04:45:45.775365 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.756590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.22.2.130:32837)
0808 04:45:45.758346 (+ 1756us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.758361 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.758711 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.759120 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.759123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.759333 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.759896 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.759916 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.761154 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.761160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.763005 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.763020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.763856 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.763862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.764214 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.765156 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.765183 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.768482 (+ 3299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.771287 (+ 2805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.771296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.771623 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.771971 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.771974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.771976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.772092 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.772923 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.772925 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.772997 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.773754 (+ 757us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.774272 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1690,"mutex_wait_us":515,"spinlock_wait_cycles":54528,"thread_start_us":94,"threads_started":1}
I20260808 04:45:45.776854 839 heartbeater.cc:347] Connected to a master server at 127.22.2.189:39161
I20260808 04:45:45.776912 836 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38987
I20260808 04:45:45.776921 837 heartbeater.cc:347] Connected to a master server at 127.22.2.188:35545
I20260808 04:45:45.777123 837 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.777123 839 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.777123 836 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.777791 839 heartbeater.cc:511] Master 127.22.2.189:39161 requested a full tablet report, sending...
I20260808 04:45:45.777842 837 heartbeater.cc:511] Master 127.22.2.188:35545 requested a full tablet report, sending...
I20260808 04:45:45.777793 836 heartbeater.cc:511] Master 127.22.2.190:38987 requested a full tablet report, sending...
I20260808 04:45:45.778973 32593 ts_manager.cc:194] Registered new tserver with Master: d8c1979ae5fb4ef0a97987256b08aa2c (127.22.2.130:46229)
I20260808 04:45:45.778977 391 ts_manager.cc:194] Registered new tserver with Master: d8c1979ae5fb4ef0a97987256b08aa2c (127.22.2.130:46229)
I20260808 04:45:45.779788 31716 ts_manager.cc:194] Re-registered known tserver with Master: d8c1979ae5fb4ef0a97987256b08aa2c (127.22.2.130:46229)
I20260808 04:45:45.780340 31716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:53009
I20260808 04:45:45.826781 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.822841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35545 (local address 127.0.0.1:45648)
0808 04:45:45.823092 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.823115 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.823238 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.823561 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.823564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.823577 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:45.823673 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.823678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.824748 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.824751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.825515 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.825525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.826440 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.826443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.826516 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.826539 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.826588 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.826652 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260808 04:45:45.826838 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.822911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45648 (local address 127.22.2.188:35545)
0808 04:45:45.823055 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.823061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.823210 (+ 149us) server_negotiation.cc:407] Connection header received
0808 04:45:45.823347 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.823351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.823425 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.823544 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:45.823780 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.824642 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.825649 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.826313 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.826429 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.826590 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.826639 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.826709 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:45:45.827615 401 raft_consensus.cc:1203] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 04:45:45.828078 446 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2416c14ab864738b27a3a0891a4b474. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:45.828255 446 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: This master's current role is: LEARNER
W20260808 04:45:45.898954 849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:45.899209 849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:45.899251 849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:45.903098 849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:45.903153 849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:45.903208 849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:45.908774 849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:41819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=40503
--tserver_master_addrs=127.22.2.190:38987,127.22.2.189:39161,127.22.2.188:35545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:45.909914 849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:45.910903 849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:45.920655 861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.920663 856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:45.920663 855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:45.921185 849 server_base.cc:1034] running on GCE node
I20260808 04:45:45.921587 849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:45.922282 849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:45.923473 849 hybrid_clock.cc:648] HybridClock initialized: now 1786164345923426 us; error 57 us; skew 500 ppm
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:45.926651 849 init.cc:377] Logged in from keytab as oryx/127.22.2.131@KRBTEST.COM (short username oryx)
I20260808 04:45:45.927454 849 webserver.cc:460] Webserver started at http://127.22.2.131:40503/ using document root <none> and password file <none>
I20260808 04:45:45.927992 849 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:45.928067 849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:45.932005 849 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:45.934460 871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.935854 849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:45.935994 849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "de3aeaed68624f448b39563148adfcbe"
format_stamp: "Formatted at 2026-08-08 04:45:44 on dist-test-slave-04bb"
server_key: "4c985d522278828e7ef46720d10e3bf7"
server_key_iv: "93bbaf7d096859a6b0ced446e89c3f6d"
server_key_version: "encryptionkey@0"
I20260808 04:45:45.936596 849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:45.949167 849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:45.952679 849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:45.952883 849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:45.953480 849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:45.954393 849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:45.954442 849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.954507 849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:45.954537 849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:45.963840 849 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:41819
I20260808 04:45:45.963924 1032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:41819 every 8 connection(s)
I20260808 04:45:45.964906 849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:45.972800 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 849
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
Aug 08 04:45:45 dist-test-slave-04bb krb5kdc[31653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164345, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 04:45:45.985358 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.966990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:60687 (local address 127.22.2.189:39161)
0808 04:45:45.967138 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.967143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.968277 (+ 1134us) server_negotiation.cc:407] Connection header received
0808 04:45:45.969287 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.969291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.969354 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.969472 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.971042 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.971951 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974212 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.974548 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974636 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.979362 (+ 4726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.979366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.979370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.979395 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.981446 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.983597 (+ 2151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.983601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.983603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.983694 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.984070 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.984075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.984076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.984301 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.984407 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.984964 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.985127 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 04:45:45.985405 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.968598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:38757 (local address 127.22.2.190:38987)
0808 04:45:45.968760 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.968766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.968786 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:45:45.969362 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.969365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.969418 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.969474 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.972120 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.972814 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974261 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.974600 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974686 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.979557 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.979563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.979566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.979598 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.982236 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.983696 (+ 1460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.983703 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.983708 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.983809 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.984137 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.984144 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.984150 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.984378 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.984475 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.985105 (+ 630us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.985239 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 04:45:45.986460 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.967340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.22.2.131:60687)
0808 04:45:45.968172 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.968188 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.969059 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.969669 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.969676 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.970219 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.970892 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.970906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.972148 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.972152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.974066 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.974077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.974817 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.975105 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.975950 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.975983 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.979218 (+ 3235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.983196 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.983200 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.983202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.983473 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.983819 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.983825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.983828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.983925 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.984503 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.984506 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.984663 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.985656 (+ 993us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.985818 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":593,"spinlock_wait_cycles":5120,"thread_start_us":153,"threads_started":1}
I20260808 04:45:45.986774 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.967340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35545 (local address 127.22.2.131:50537)
0808 04:45:45.967881 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.967935 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.969104 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.969741 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.969747 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.970206 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.970907 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.970918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.972148 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.972152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.974066 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.974077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.974812 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.975146 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.976143 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.976161 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.979366 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.982518 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.982521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.982524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.982913 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.983463 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.983470 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.983472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.983675 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.984266 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.984273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.984640 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.984868 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.985261 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":512,"spinlock_wait_cycles":90624,"thread_start_us":179,"threads_started":1}
I20260808 04:45:45.987584 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.968430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:38757)
0808 04:45:45.968673 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:45.968686 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:45.969101 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:45.969682 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:45.969742 (+ 60us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:45.970206 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:45.970911 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.970918 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.972995 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.972999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.974090 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:45.974112 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.974893 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:45.975126 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.976044 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:45.976062 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:45.978867 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:45.982372 (+ 3505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.982379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.982396 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.983561 (+ 1165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.983926 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:45.983929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:45.983931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:45.984019 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:45.984545 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:45.984549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:45.984674 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 04:45:45.984868 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 04:45:45.987386 (+ 2518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":2373,"spinlock_wait_cycles":34560,"thread_start_us":101,"threads_started":1}
I20260808 04:45:45.987702 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.966919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:50537 (local address 127.22.2.188:35545)
0808 04:45:45.967096 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.967112 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.968035 (+ 923us) server_negotiation.cc:407] Connection header received
0808 04:45:45.969334 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.969337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.969407 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.969472 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.971092 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.971958 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974215 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.974565 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.974656 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.979924 (+ 5268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.979928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.979931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.979964 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.982369 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.983223 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.983230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.983234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.983338 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.983793 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.983836 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.983841 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.984089 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.984206 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.987403 (+ 3197us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.987522 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 04:45:45.988358 1035 heartbeater.cc:347] Connected to a master server at 127.22.2.188:35545
I20260808 04:45:45.988665 1035 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.988999 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.977483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37954 (local address 127.22.2.189:39161)
0808 04:45:45.978186 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.978191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.978597 (+ 406us) server_negotiation.cc:407] Connection header received
0808 04:45:45.978655 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.978658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.978710 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.978805 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.979914 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.980798 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.982069 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.982438 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.982539 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.984571 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.984578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.984580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.984605 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.986376 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.986912 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.986917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.986920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.987016 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.987615 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.987619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.987624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.987867 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.987997 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.988638 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.988775 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":87,"threads_started":1}
I20260808 04:45:45.989078 1034 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38987
I20260808 04:45:45.989158 1034 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.989270 1035 heartbeater.cc:511] Master 127.22.2.188:35545 requested a full tablet report, sending...
I20260808 04:45:45.989295 1034 heartbeater.cc:511] Master 127.22.2.190:38987 requested a full tablet report, sending...
I20260808 04:45:45.988359 1037 heartbeater.cc:347] Connected to a master server at 127.22.2.189:39161
I20260808 04:45:45.989902 1037 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:45.990056 1037 heartbeater.cc:511] Master 127.22.2.189:39161 requested a full tablet report, sending...
I20260808 04:45:45.990666 31716 ts_manager.cc:194] Re-registered known tserver with Master: de3aeaed68624f448b39563148adfcbe (127.22.2.131:41819)
I20260808 04:45:45.990738 391 ts_manager.cc:194] Registered new tserver with Master: de3aeaed68624f448b39563148adfcbe (127.22.2.131:41819)
I20260808 04:45:45.991386 31716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:38757
I20260808 04:45:45.991940 32593 ts_manager.cc:194] Registered new tserver with Master: de3aeaed68624f448b39563148adfcbe (127.22.2.131:41819)
I20260808 04:45:45.993955 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.976739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37572 (local address 127.22.2.190:38987)
0808 04:45:45.977510 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.977516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.977947 (+ 431us) server_negotiation.cc:407] Connection header received
0808 04:45:45.977997 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.978002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.978064 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.978164 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.980865 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.981849 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.986223 (+ 4374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.986627 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.986762 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.989587 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.989591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.989594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.989621 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.991935 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.992478 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.992482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.992484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.992617 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.993000 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.993003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.993006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.993242 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.993404 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.993677 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.993811 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":84,"threads_started":1}
I20260808 04:45:45.996315 1050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:45.975514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45654 (local address 127.22.2.188:35545)
0808 04:45:45.975798 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:45.975802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:45.976533 (+ 731us) server_negotiation.cc:407] Connection header received
0808 04:45:45.976616 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:45.976623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:45.976680 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:45.976769 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:45.978043 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.978907 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.988176 (+ 9269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:45.989927 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:45.990470 (+ 543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:45.992465 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:45.992469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:45.992472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:45.992510 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:45.994402 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.994870 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.994874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.994875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.994968 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:45.995318 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:45.995321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:45.995323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:45.995575 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:45.995695 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:45.995985 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 04:45:45.996113 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":140,"threads_started":1}
I20260808 04:45:45.997218 31716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:46.000100 31716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:46.002751 339 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: true } } }
I20260808 04:45:46.002900 339 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:46.006572 1059 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: attempting to promote NON_VOTER eaec38a20b3845e0ad056adafb8f6afe to VOTER
I20260808 04:45:46.006891 1059 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } }
I20260808 04:45:46.008453 32613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Refusing update from remote peer d2416c14ab864738b27a3a0891a4b474: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 04:45:46.008837 401 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 LEARNER]: Refusing update from remote peer d2416c14ab864738b27a3a0891a4b474: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 04:45:46.009017 339 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 04:45:46.009353 1062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 04:45:46.011101 1059 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:46.011656 32613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:46.012526 1059 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:46.012668 1059 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:46.012377 401 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:46.012813 1059 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:46.012898 1062 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:46.013011 1062 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:46.012903 1059 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:46.013585 1076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:46.013765 1076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:45:46.014506 446 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2416c14ab864738b27a3a0891a4b474" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:46.014618 446 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:45:46.021255 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.015924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:46229 (local address 127.0.0.1:57140)
0808 04:45:46.016060 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:46.016075 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:46.016158 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:46.016950 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:46.016952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:46.016969 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:46.017090 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.017099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.018977 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.018980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.019698 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.019707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.020643 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.020647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.020862 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.020934 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 04:45:46.021029 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 04:45:46.021126 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":81}
I20260808 04:45:46.021317 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.016058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57140 (local address 127.22.2.130:46229)
0808 04:45:46.016374 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.016388 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.016414 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:45:46.016497 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.016502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.016779 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.016972 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:46.017195 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.018863 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.019828 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.020475 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.020642 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.020834 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.021067 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.021163 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":128,"threads_started":1}
I20260808 04:45:46.024606 746 tablet_service.cc:1467] Processing CreateTablet for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc (DEFAULT_TABLE table=test-table [id=08c242df24b8489d810f86629eb67cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:46.025760 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.017328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45035 (local address 127.0.0.1:53346)
0808 04:45:46.019104 (+ 1776us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:46.019119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:46.019299 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:46.020641 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:46.020644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:46.020657 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:46.020883 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.020893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.022423 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.022430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.024026 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.024039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.025260 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.025264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.025368 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.025397 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 04:45:46.025506 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 04:45:46.025591 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1707,"mutex_wait_us":71,"spinlock_wait_cycles":2816,"thread_start_us":224,"threads_started":1}
I20260808 04:45:46.025825 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.017519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.22.2.129:45035)
0808 04:45:46.019941 (+ 2422us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.019962 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.019984 (+ 22us) server_negotiation.cc:407] Connection header received
0808 04:45:46.020056 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.020061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.020324 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.021026 (+ 702us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:46.021171 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.022276 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.024267 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.025139 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.025253 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.025405 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.025539 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.025646 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2319,"thread_start_us":80,"threads_started":1}
I20260808 04:45:46.025871 746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:46.025950 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.017612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:41819 (local address 127.0.0.1:35256)
0808 04:45:46.019439 (+ 1827us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:46.019448 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:46.019699 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:46.020621 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:46.020624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:46.020644 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:46.020898 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.020903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.022520 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.022523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.024026 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.024039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.025487 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.025491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.025596 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.025635 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 04:45:46.025718 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 04:45:46.025794 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1781,"mutex_wait_us":194,"thread_start_us":109,"threads_started":1}
I20260808 04:45:46.026381 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.017788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35256 (local address 127.22.2.131:41819)
0808 04:45:46.018349 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.018362 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.019685 (+ 1323us) server_negotiation.cc:407] Connection header received
0808 04:45:46.019817 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.019823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.020035 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.020357 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:46.021025 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.022404 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.024197 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.025129 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.025253 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.025405 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.026055 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.026195 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":324,"threads_started":1}
I20260808 04:45:46.028951 556 tablet_service.cc:1467] Processing CreateTablet for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc (DEFAULT_TABLE table=test-table [id=08c242df24b8489d810f86629eb67cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:46.030009 556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:46.033598 943 tablet_service.cc:1467] Processing CreateTablet for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc (DEFAULT_TABLE table=test-table [id=08c242df24b8489d810f86629eb67cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:46.034192 1086 tablet_bootstrap.cc:492] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Bootstrap starting.
I20260808 04:45:46.034672 943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:46.036708 1086 tablet_bootstrap.cc:654] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:46.037575 1086 log.cc:824] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:46.039651 1086 tablet_bootstrap.cc:492] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: No bootstrap required, opened a new log
I20260808 04:45:46.039877 1086 ts_tablet_manager.cc:1397] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260808 04:45:46.044185 1090 tablet_bootstrap.cc:492] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Bootstrap starting.
I20260808 04:45:46.044884 1086 raft_consensus.cc:348] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.045182 1086 raft_consensus.cc:374] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:46.045244 1086 raft_consensus.cc:729] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c1979ae5fb4ef0a97987256b08aa2c, State: Initialized, Role: FOLLOWER
I20260808 04:45:46.045794 1086 consensus_queue.cc:260] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.046550 1090 tablet_bootstrap.cc:654] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:46.046886 1086 ts_tablet_manager.cc:1428] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260808 04:45:46.047278 836 heartbeater.cc:503] Master 127.22.2.190:38987 was elected leader, sending a full tablet report...
I20260808 04:45:46.047503 1090 log.cc:824] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:46.049180 1092 tablet_bootstrap.cc:492] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Bootstrap starting.
I20260808 04:45:46.051844 1092 tablet_bootstrap.cc:654] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:46.052942 1092 log.cc:824] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:46.055001 1092 tablet_bootstrap.cc:492] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: No bootstrap required, opened a new log
I20260808 04:45:46.055231 1092 ts_tablet_manager.cc:1397] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:45:46.056957 1090 tablet_bootstrap.cc:492] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: No bootstrap required, opened a new log
I20260808 04:45:46.057165 1090 ts_tablet_manager.cc:1397] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260808 04:45:46.058869 1092 raft_consensus.cc:348] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.059082 1092 raft_consensus.cc:374] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:46.059170 1092 raft_consensus.cc:729] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e130596245467ab759328b94fd2ed2, State: Initialized, Role: FOLLOWER
I20260808 04:45:46.059624 1092 consensus_queue.cc:260] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.060299 649 heartbeater.cc:503] Master 127.22.2.190:38987 was elected leader, sending a full tablet report...
I20260808 04:45:46.060711 1092 ts_tablet_manager.cc:1428] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260808 04:45:46.061641 1090 raft_consensus.cc:348] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.061921 1090 raft_consensus.cc:374] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:46.061990 1090 raft_consensus.cc:729] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3aeaed68624f448b39563148adfcbe, State: Initialized, Role: FOLLOWER
I20260808 04:45:46.062433 1090 consensus_queue.cc:260] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.063153 1034 heartbeater.cc:503] Master 127.22.2.190:38987 was elected leader, sending a full tablet report...
I20260808 04:45:46.063527 1090 ts_tablet_manager.cc:1428] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 04:45:46.216905 1038 tablet.cc:2369] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:45:46.255326 840 tablet.cc:2369] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:45:46.264950 654 tablet.cc:2369] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:46.405149 1097 raft_consensus.cc:482] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:46.405375 1097 raft_consensus.cc:504] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.406675 1097 leader_election.cc:288] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8c1979ae5fb4ef0a97987256b08aa2c (127.22.2.130:46229), b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035)
I20260808 04:45:46.410990 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.407425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:59659 (local address 127.22.2.129:45035)
0808 04:45:46.407557 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.407560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.407572 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:45:46.407646 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.407649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.407701 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.407803 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:46.408115 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.408708 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.409993 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.410367 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.410464 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.410505 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.410769 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.410855 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 04:45:46.410990 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.406909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:46229 (local address 127.22.2.131:57119)
0808 04:45:46.407094 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:46.407114 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:46.407265 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:46.407665 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:46.407668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:46.407685 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:46.407846 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.407854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.408881 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.408885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.409820 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.409830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.410569 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.410572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.410666 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.410683 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 04:45:46.410796 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 04:45:46.410859 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":6}
I20260808 04:45:46.411013 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.407286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45035 (local address 127.22.2.131:59659)
0808 04:45:46.407441 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:46.407456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:46.407528 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:46.407835 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:46.407837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:46.407853 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:46.407995 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.408004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.408872 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.408875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.409829 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:46.409839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.410494 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.410498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:46.410633 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.410651 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 04:45:46.410716 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 04:45:46.410815 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":77}
I20260808 04:45:46.411262 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.407040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:57119 (local address 127.22.2.130:46229)
0808 04:45:46.407210 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.407216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.407232 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:45:46.407485 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.407492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.407552 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.407637 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:46.407978 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.408720 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.409939 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.410365 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.410504 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.410555 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.411069 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.411154 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 04:45:46.411916 772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd8aae7fb9f4a619c54c90a7ea11ebc" candidate_uuid: "de3aeaed68624f448b39563148adfcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" is_pre_election: true
I20260808 04:45:46.411926 587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd8aae7fb9f4a619c54c90a7ea11ebc" candidate_uuid: "de3aeaed68624f448b39563148adfcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e130596245467ab759328b94fd2ed2" is_pre_election: true
I20260808 04:45:46.412223 587 raft_consensus.cc:2454] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de3aeaed68624f448b39563148adfcbe in term 0.
I20260808 04:45:46.412226 772 raft_consensus.cc:2454] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de3aeaed68624f448b39563148adfcbe in term 0.
I20260808 04:45:46.412720 875 leader_election.cc:302] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e130596245467ab759328b94fd2ed2, de3aeaed68624f448b39563148adfcbe; no voters:
I20260808 04:45:46.412997 1097 raft_consensus.cc:2781] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:46.413146 1097 raft_consensus.cc:482] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:46.413257 1097 raft_consensus.cc:3037] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:46.414366 1097 raft_consensus.cc:504] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.414736 1097 leader_election.cc:288] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [CANDIDATE]: Term 1 election: Requested vote from peers d8c1979ae5fb4ef0a97987256b08aa2c (127.22.2.130:46229), b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035)
I20260808 04:45:46.415019 772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd8aae7fb9f4a619c54c90a7ea11ebc" candidate_uuid: "de3aeaed68624f448b39563148adfcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c"
I20260808 04:45:46.415140 772 raft_consensus.cc:3037] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:46.416337 772 raft_consensus.cc:2454] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de3aeaed68624f448b39563148adfcbe in term 1.
I20260808 04:45:46.416621 875 leader_election.cc:302] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c1979ae5fb4ef0a97987256b08aa2c, de3aeaed68624f448b39563148adfcbe; no voters:
I20260808 04:45:46.416615 587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd8aae7fb9f4a619c54c90a7ea11ebc" candidate_uuid: "de3aeaed68624f448b39563148adfcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e130596245467ab759328b94fd2ed2"
I20260808 04:45:46.416760 587 raft_consensus.cc:3037] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:46.416863 1097 raft_consensus.cc:2781] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:46.417097 1097 raft_consensus.cc:686] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 1 LEADER]: Becoming Leader. State: Replica: de3aeaed68624f448b39563148adfcbe, State: Running, Role: LEADER
I20260808 04:45:46.417742 1097 consensus_queue.cc:237] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } }
I20260808 04:45:46.418030 587 raft_consensus.cc:2454] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de3aeaed68624f448b39563148adfcbe in term 1.
I20260808 04:45:46.421403 31716 catalog_manager.cc:5495] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe reported cstate change: term changed from 0 to 1, leader changed from <none> to de3aeaed68624f448b39563148adfcbe (127.22.2.131). New cstate: current_term: 1 leader_uuid: "de3aeaed68624f448b39563148adfcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } health_report { overall_health: HEALTHY } } }
I20260808 04:45:46.474385 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.470628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.22.2.131:41819)
0808 04:45:46.470796 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.470800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.470817 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:45:46.470938 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.470940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.471000 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.471119 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:46.471379 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.471890 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.472822 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.473043 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.473169 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.473515 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:46.474000 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:46.474133 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.474177 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.474237 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 04:45:46.480104 587 raft_consensus.cc:1261] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 1 FOLLOWER]: Refusing update from remote peer de3aeaed68624f448b39563148adfcbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:46.480104 772 raft_consensus.cc:1261] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 FOLLOWER]: Refusing update from remote peer de3aeaed68624f448b39563148adfcbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:46.480937 1099 consensus_queue.cc:1035] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:46.481101 1097 consensus_queue.cc:1035] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:46.489849 1104 mvcc.cc:205] Tried to move back new op lower bound from 7316129163175329792 to 7316129162930249728. Current Snapshot: MvccSnapshot[applied={T|T < 7316129163175329792}]
I20260808 04:45:46.491552 1102 mvcc.cc:205] Tried to move back new op lower bound from 7316129163175329792 to 7316129162930249728. Current Snapshot: MvccSnapshot[applied={T|T < 7316129163175329792}]
I20260808 04:45:46.492657 1103 mvcc.cc:205] Tried to move back new op lower bound from 7316129163175329792 to 7316129162930249728. Current Snapshot: MvccSnapshot[applied={T|T < 7316129163175329792}]
I20260808 04:45:46.500912 31717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:46.501092 31717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:46.506515 31717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2416c14ab864738b27a3a0891a4b474: Sending DeleteTablet for 3 replicas of tablet ffd8aae7fb9f4a619c54c90a7ea11ebc
I20260808 04:45:46.507411 746 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:46 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:57140
I20260808 04:45:46.507586 556 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:46 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:53346
I20260808 04:45:46.507710 943 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd8aae7fb9f4a619c54c90a7ea11ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:46 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:35256
I20260808 04:45:46.508093 1114 tablet_replica.cc:330] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: stopping tablet replica
I20260808 04:45:46.508240 1115 tablet_replica.cc:330] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: stopping tablet replica
I20260808 04:45:46.508595 1115 raft_consensus.cc:2229] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:46.508589 1114 raft_consensus.cc:2229] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:46.509045 1115 raft_consensus.cc:2258] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:46.509074 1114 raft_consensus.cc:2258] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:46.510638 1114 ts_tablet_manager.cc:1905] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:46.510780 1115 ts_tablet_manager.cc:1905] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:46.511451 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 31674
I20260808 04:45:46.520778 1115 ts_tablet_manager.cc:1918] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:46.520913 1115 log.cc:1196] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/ffd8aae7fb9f4a619c54c90a7ea11ebc
I20260808 04:45:46.521293 1115 ts_tablet_manager.cc:1939] T ffd8aae7fb9f4a619c54c90a7ea11ebc P de3aeaed68624f448b39563148adfcbe: Deleting consensus metadata
I20260808 04:45:46.527336 1114 ts_tablet_manager.cc:1918] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:46.527463 1114 log.cc:1196] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/ffd8aae7fb9f4a619c54c90a7ea11ebc
I20260808 04:45:46.527853 1114 ts_tablet_manager.cc:1939] T ffd8aae7fb9f4a619c54c90a7ea11ebc P b8e130596245467ab759328b94fd2ed2: Deleting consensus metadata
W20260808 04:45:46.529400 486 connection.cc:535] server connection from 127.0.0.1:53346 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 04:45:46.529395 873 connection.cc:535] server connection from 127.0.0.1:35256 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 04:45:46.530112 1113 tablet_replica.cc:330] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: stopping tablet replica
I20260808 04:45:46.530505 1113 raft_consensus.cc:2229] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:46.530967 1113 raft_consensus.cc:2258] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:46.533119 1113 ts_tablet_manager.cc:1905] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:46.536139 1113 ts_tablet_manager.cc:1918] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:46.536259 1113 log.cc:1196] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/ffd8aae7fb9f4a619c54c90a7ea11ebc
I20260808 04:45:46.536703 1113 ts_tablet_manager.cc:1939] T ffd8aae7fb9f4a619c54c90a7ea11ebc P d8c1979ae5fb4ef0a97987256b08aa2c: Deleting consensus metadata
W20260808 04:45:46.537989 684 connection.cc:441] server connection from 127.0.0.1:57140 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57140 (request call id 1) could send its response
I20260808 04:45:46.541011 1119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.509602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.22.2.189:39161)
0808 04:45:46.529897 (+ 20295us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.529902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.530149 (+ 247us) server_negotiation.cc:407] Connection header received
0808 04:45:46.530205 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.530208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.530264 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.530360 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:46.531748 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.532957 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.534264 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.534621 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.534728 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.536857 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:46.536861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:46.536864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:46.536892 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:46.538812 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:46.539338 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:46.539342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:46.539344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:46.539433 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:46.539746 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:46.539749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:46.539751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:46.540012 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:46.540139 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.540637 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.540784 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20213,"thread_start_us":123,"threads_started":1}
I20260808 04:45:46.547768 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.535902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.22.2.189:39161)
0808 04:45:46.537024 (+ 1122us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.537028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.537331 (+ 303us) server_negotiation.cc:407] Connection header received
0808 04:45:46.537369 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.537372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.537416 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.537495 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:46.538877 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.539696 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.540860 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.541186 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.541293 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.542943 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:46.542950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:46.542956 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:46.542984 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:46.545409 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:46.546151 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:46.546155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:46.546157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:46.546248 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:46.546553 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:46.546556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:46.546558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:46.546824 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:46.546944 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.547469 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.547582 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":77,"threads_started":1}
I20260808 04:45:46.549468 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.538381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.22.2.188:35545)
0808 04:45:46.538831 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.538837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.538852 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:46.538893 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.538896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.538945 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.540231 (+ 1286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:46.540342 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.541238 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.542464 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.542863 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.543292 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.544693 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:46.544697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:46.544700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:46.544728 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:46.547124 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:46.547665 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:46.547667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:46.547669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:46.547762 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:46.548127 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:46.548133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:46.548140 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:46.548375 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:46.549167 (+ 792us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.549245 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.549352 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":69,"threads_started":1}
I20260808 04:45:46.551645 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:46.509706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45670 (local address 127.22.2.188:35545)
0808 04:45:46.509955 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:46.509960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:46.542837 (+ 32877us) server_negotiation.cc:407] Connection header received
0808 04:45:46.543009 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:46.543013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:46.543057 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:46.543115 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:46.543832 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.544649 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.545758 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:46.546145 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:46.546241 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:46.547897 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:46.547901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:46.547903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:46.547927 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:46.549727 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:46.550322 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:46.550326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:46.550328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:46.550412 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:46.550727 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:46.550730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:46.550732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:46.550940 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:46.551072 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:46.551257 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 04:45:46.551373 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":99,"threads_started":1}
I20260808 04:45:47.493212 1129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:47.492669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:42039)
0808 04:45:47.492973 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:47.493080 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":103,"threads_started":1}
W20260808 04:45:47.493613 649 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:38987 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
I20260808 04:45:47.496460 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:47.495995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:44371)
0808 04:45:47.496268 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:47.496359 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":65,"threads_started":1}
I20260808 04:45:47.496841 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:47.496407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:36321)
0808 04:45:47.496647 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:47.496747 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
W20260808 04:45:47.496871 836 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:38987 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
W20260808 04:45:47.497219 1034 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:38987 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
I20260808 04:45:48.006210 1134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2416c14ab864738b27a3a0891a4b474)
I20260808 04:45:48.006517 1134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } }
I20260808 04:45:48.006624 1137 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2416c14ab864738b27a3a0891a4b474)
I20260808 04:45:48.006875 1137 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } }
I20260808 04:45:48.007690 1134 leader_election.cc:288] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987), eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188:35545)
I20260808 04:45:48.008388 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.007914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38028)
0808 04:45:48.008177 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.008267 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
I20260808 04:45:48.008428 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.008068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38038)
0808 04:45:48.008272 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.008340 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":60,"threads_started":1}
W20260808 04:45:48.008649 373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
W20260808 04:45:48.008738 32587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
I20260808 04:45:48.009609 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.009348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38046)
0808 04:45:48.009490 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.009545 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 04:45:48.009609 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.009337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38048)
0808 04:45:48.009490 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.009545 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260808 04:45:48.009735 32587 leader_election.cc:334] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987): Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
W20260808 04:45:48.009925 373 leader_election.cc:334] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987): Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
I20260808 04:45:48.010082 1137 leader_election.cc:288] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987), 58f82dfd6d4b4e4f93adc6e228615efc (127.22.2.189:39161)
I20260808 04:45:48.013231 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.007956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35545 (local address 127.0.0.1:45686)
0808 04:45:48.008364 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.008384 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:48.008562 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:48.009955 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:48.009959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:48.009982 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:48.010104 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:48.010112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:48.011044 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:48.011049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:48.011933 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:48.011943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:48.012703 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:48.012707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:48.012837 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:48.012898 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 04:45:48.012997 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 04:45:48.013077 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":36,"thread_start_us":61,"threads_started":1}
I20260808 04:45:48.013229 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.008306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.22.2.188:35545)
0808 04:45:48.008689 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:48.008698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:48.008736 (+ 38us) server_negotiation.cc:407] Connection header received
0808 04:45:48.009115 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:48.009119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:48.009451 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:48.009539 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:48.010302 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:48.010918 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:48.012067 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:48.012512 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:48.012714 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:48.012781 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:48.013024 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 04:45:48.013103 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":310,"thread_start_us":96,"threads_started":1}
I20260808 04:45:48.013365 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.009803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:39161 (local address 127.0.0.1:37990)
0808 04:45:48.009931 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.009948 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:48.010048 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:48.010461 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:48.010464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:48.010482 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:48.010585 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:48.010589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:48.011320 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:48.011326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:48.012111 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:48.012121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:48.012838 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:48.012842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:48.012933 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:48.012958 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 04:45:48.013080 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 04:45:48.013162 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":8}
I20260808 04:45:48.013718 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.009867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.22.2.189:39161)
0808 04:45:48.010244 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:48.010252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:48.010263 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:45:48.010299 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:48.010302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:48.010349 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:48.010447 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:48.010694 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:48.011208 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:48.012240 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:48.012706 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:48.012889 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:48.012946 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:48.013500 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 04:45:48.013584 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":306,"thread_start_us":100,"threads_started":1}
I20260808 04:45:48.013969 401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eaec38a20b3845e0ad056adafb8f6afe" is_pre_election: true
I20260808 04:45:48.014146 32613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eaec38a20b3845e0ad056adafb8f6afe" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" is_pre_election: true
I20260808 04:45:48.014283 401 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58f82dfd6d4b4e4f93adc6e228615efc in term 1.
I20260808 04:45:48.014346 32613 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaec38a20b3845e0ad056adafb8f6afe in term 1.
I20260808 04:45:48.015211 373 leader_election.cc:302] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58f82dfd6d4b4e4f93adc6e228615efc, eaec38a20b3845e0ad056adafb8f6afe; no voters: d2416c14ab864738b27a3a0891a4b474
I20260808 04:45:48.015313 32585 leader_election.cc:302] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58f82dfd6d4b4e4f93adc6e228615efc, eaec38a20b3845e0ad056adafb8f6afe; no voters: d2416c14ab864738b27a3a0891a4b474
I20260808 04:45:48.015435 1137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 04:45:48.015492 1137 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2416c14ab864738b27a3a0891a4b474)
I20260808 04:45:48.015532 1137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:45:48.015705 1134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 04:45:48.015774 1134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2416c14ab864738b27a3a0891a4b474)
I20260808 04:45:48.015836 1134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:45:48.016705 1137 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } }
I20260808 04:45:48.017045 1137 leader_election.cc:288] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 2 election: Requested vote from peers d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987), 58f82dfd6d4b4e4f93adc6e228615efc (127.22.2.189:39161)
I20260808 04:45:48.017092 1134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } }
I20260808 04:45:48.017515 1134 leader_election.cc:288] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [CANDIDATE]: Term 2 election: Requested vote from peers d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987), eaec38a20b3845e0ad056adafb8f6afe (127.22.2.188:35545)
I20260808 04:45:48.017582 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.017218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38068)
0808 04:45:48.017453 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.017510 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260808 04:45:48.017917 32613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eaec38a20b3845e0ad056adafb8f6afe" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "58f82dfd6d4b4e4f93adc6e228615efc"
I20260808 04:45:48.017974 401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eaec38a20b3845e0ad056adafb8f6afe"
I20260808 04:45:48.018087 32613 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eaec38a20b3845e0ad056adafb8f6afe in current term 2: Already voted for candidate 58f82dfd6d4b4e4f93adc6e228615efc in this term.
I20260808 04:45:48.018115 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.017625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38080)
0808 04:45:48.017989 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.018048 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291}
I20260808 04:45:48.018139 401 raft_consensus.cc:2379] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 58f82dfd6d4b4e4f93adc6e228615efc in current term 2: Already voted for candidate eaec38a20b3845e0ad056adafb8f6afe in this term.
I20260808 04:45:48.018330 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.018071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38088)
0808 04:45:48.018215 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.018274 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260808 04:45:48.018497 373 leader_election.cc:334] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987): Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
I20260808 04:45:48.018741 373 leader_election.cc:302] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eaec38a20b3845e0ad056adafb8f6afe; no voters: 58f82dfd6d4b4e4f93adc6e228615efc, d2416c14ab864738b27a3a0891a4b474
I20260808 04:45:48.018795 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.018597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38104)
0808 04:45:48.018702 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.018750 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 04:45:48.018914 32587 leader_election.cc:334] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987): Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
I20260808 04:45:48.018959 1137 raft_consensus.cc:2726] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 04:45:48.019131 32585 leader_election.cc:302] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58f82dfd6d4b4e4f93adc6e228615efc; no voters: d2416c14ab864738b27a3a0891a4b474, eaec38a20b3845e0ad056adafb8f6afe
I20260808 04:45:48.019294 1134 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 04:45:48.494752 1148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.494274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:42643)
0808 04:45:48.494565 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.494634 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":123,"threads_started":1}
I20260808 04:45:48.498054 1149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.497486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:47169)
0808 04:45:48.497874 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.497925 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":78,"threads_started":1}
I20260808 04:45:48.498288 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:48.497827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:50041)
0808 04:45:48.498107 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:48.498177 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
I20260808 04:45:49.496368 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.495819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:54023)
0808 04:45:49.496152 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.496230 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":123,"threads_started":1}
I20260808 04:45:49.499353 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.498881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:47431)
0808 04:45:49.499177 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.499248 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":76,"threads_started":1}
I20260808 04:45:49.499774 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.499037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:54553)
0808 04:45:49.499638 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.499695 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":72,"threads_started":1}
I20260808 04:45:49.525710 1156 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:49.525861 1156 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } }
I20260808 04:45:49.526358 1156 leader_election.cc:288] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987), 58f82dfd6d4b4e4f93adc6e228615efc (127.22.2.189:39161)
I20260808 04:45:49.526856 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.526501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38318)
0808 04:45:49.526710 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.526776 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":55,"threads_started":1}
I20260808 04:45:49.527040 32613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eaec38a20b3845e0ad056adafb8f6afe" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" is_pre_election: true
I20260808 04:45:49.527202 32613 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaec38a20b3845e0ad056adafb8f6afe in term 2.
I20260808 04:45:49.527606 373 leader_election.cc:302] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f82dfd6d4b4e4f93adc6e228615efc, eaec38a20b3845e0ad056adafb8f6afe; no voters:
I20260808 04:45:49.527709 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.527483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38332)
0808 04:45:49.527574 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.527624 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 04:45:49.527784 1156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260808 04:45:49.527787 373 leader_election.cc:334] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987): Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
I20260808 04:45:49.527865 1156 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:49.527980 1156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 2 FOLLOWER]: Advancing to term 3
I20260808 04:45:49.529275 1156 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } }
I20260808 04:45:49.529589 1156 leader_election.cc:288] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 3 election: Requested vote from peers d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987), 58f82dfd6d4b4e4f93adc6e228615efc (127.22.2.189:39161)
I20260808 04:45:49.530018 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.529762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38346)
0808 04:45:49.529867 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.529950 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 04:45:49.530179 32613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eaec38a20b3845e0ad056adafb8f6afe" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "58f82dfd6d4b4e4f93adc6e228615efc"
I20260808 04:45:49.530297 32613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 2 FOLLOWER]: Advancing to term 3
I20260808 04:45:49.530629 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.530446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38358)
0808 04:45:49.530531 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.530583 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 04:45:49.530731 373 leader_election.cc:334] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987): Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
I20260808 04:45:49.531423 32613 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaec38a20b3845e0ad056adafb8f6afe in term 3.
I20260808 04:45:49.531704 373 leader_election.cc:302] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58f82dfd6d4b4e4f93adc6e228615efc, eaec38a20b3845e0ad056adafb8f6afe; no voters: d2416c14ab864738b27a3a0891a4b474
I20260808 04:45:49.531945 1156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 3 FOLLOWER]: Leader election won for term 3
I20260808 04:45:49.532124 1156 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [term 3 LEADER]: Becoming Leader. State: Replica: eaec38a20b3845e0ad056adafb8f6afe, State: Running, Role: LEADER
I20260808 04:45:49.532462 1156 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } }
I20260808 04:45:49.533842 1159 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaec38a20b3845e0ad056adafb8f6afe. Latest consensus state: current_term: 3 leader_uuid: "eaec38a20b3845e0ad056adafb8f6afe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:49.533933 1159 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:49.534191 1162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:49.537000 1162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:49.537492 1162 catalog_manager.cc:1259] Loaded cluster ID: e84031d74e684823b9b396333083bff1
I20260808 04:45:49.537562 1162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:49.538137 1162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:49.538547 1162 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Loaded TSK: 0
I20260808 04:45:49.539256 1162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:45:49.624388 391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:49.627579 391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:49.630959 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.630687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38384)
0808 04:45:49.630811 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.630871 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 04:45:49.631356 32613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [term 3 FOLLOWER]: Refusing update from remote peer eaec38a20b3845e0ad056adafb8f6afe: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 04:45:49.631820 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.631504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38392)
0808 04:45:49.631711 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.631762 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 04:45:49.631954 1156 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 04:45:49.632156 373 consensus_peers.cc:487] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe -> Peer d2416c14ab864738b27a3a0891a4b474 (127.22.2.190:38987): Couldn't send request to peer d2416c14ab864738b27a3a0891a4b474. Status: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 04:45:49.633667 1165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaec38a20b3845e0ad056adafb8f6afe. Latest consensus state: current_term: 3 leader_uuid: "eaec38a20b3845e0ad056adafb8f6afe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:49.633812 1165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:45:49.635056 1159 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "eaec38a20b3845e0ad056adafb8f6afe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:49.635210 1159 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:49.635382 1165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "eaec38a20b3845e0ad056adafb8f6afe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d2416c14ab864738b27a3a0891a4b474" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38987 } } peers { permanent_uuid: "58f82dfd6d4b4e4f93adc6e228615efc" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 39161 } attrs { promote: false } } peers { permanent_uuid: "eaec38a20b3845e0ad056adafb8f6afe" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35545 } attrs { promote: false } } }
I20260808 04:45:49.635497 1165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f82dfd6d4b4e4f93adc6e228615efc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:45:49.645987 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.641186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:46229 (local address 127.0.0.1:57144)
0808 04:45:49.641341 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.641357 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:49.641459 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:49.642355 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:49.642358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:49.642375 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:49.642495 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:49.642501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.643462 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.643466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:49.644746 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:49.644756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.645597 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.645601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:49.645704 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:49.645734 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 04:45:49.645783 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 04:45:49.645850 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":36}
I20260808 04:45:49.646358 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.641498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:41819 (local address 127.0.0.1:35274)
0808 04:45:49.641754 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.641767 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:49.641859 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:49.642738 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:49.642740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:49.642756 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:49.642857 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:49.642866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.643919 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.643922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:49.645074 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:49.645082 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.645957 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.645960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:49.646050 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:49.646075 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 04:45:49.646149 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 04:45:49.646221 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":21,"thread_start_us":108,"threads_started":1}
I20260808 04:45:49.646490 1171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.641375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.22.2.130:46229)
0808 04:45:49.642093 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:49.642098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:49.642113 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:49.642178 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:49.642183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:49.642238 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:49.642348 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:49.642620 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.643360 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.644872 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.645465 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.646182 (+ 717us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:49.646244 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:49.646280 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 04:45:49.646359 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":72,"threads_started":1}
I20260808 04:45:49.646946 1174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.641763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.22.2.131:41819)
0808 04:45:49.642497 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:49.642501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:49.642522 (+ 21us) server_negotiation.cc:407] Connection header received
0808 04:45:49.642590 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:49.642593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:49.642640 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:49.642727 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:49.643033 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.643783 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.645190 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.645821 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.645945 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:49.646008 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:49.646512 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 04:45:49.646846 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":107,"threads_started":1}
I20260808 04:45:49.647069 746 tablet_service.cc:1467] Processing CreateTablet for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac (DEFAULT_TABLE table=test-table [id=c5bf001d1c0c4fca831cda65761cf4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:49.647197 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.642077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45035 (local address 127.0.0.1:53350)
0808 04:45:49.642498 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.642511 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:49.642589 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:49.643329 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:49.643332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:49.643353 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:49.643517 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:49.643524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.644259 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.644264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:49.645605 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:49.645614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.646724 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.646727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:49.646823 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:49.646846 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 04:45:49.646919 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:45:49.646983 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":63,"thread_start_us":102,"threads_started":1}
I20260808 04:45:49.647411 746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:49.647560 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.642706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.22.2.129:45035)
0808 04:45:49.643082 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:49.643086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:49.643100 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:45:49.643148 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:49.643151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:49.643197 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:49.643298 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:49.643642 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.644143 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.645733 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:49.646482 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:49.646592 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:49.646657 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:49.647353 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 04:45:49.647442 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":65,"threads_started":1}
I20260808 04:45:49.647604 943 tablet_service.cc:1467] Processing CreateTablet for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac (DEFAULT_TABLE table=test-table [id=c5bf001d1c0c4fca831cda65761cf4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:49.647920 943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:49.648085 556 tablet_service.cc:1467] Processing CreateTablet for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac (DEFAULT_TABLE table=test-table [id=c5bf001d1c0c4fca831cda65761cf4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:49.648375 556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:49.649893 1180 tablet_bootstrap.cc:492] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: Bootstrap starting.
I20260808 04:45:49.650517 1181 tablet_bootstrap.cc:492] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: Bootstrap starting.
I20260808 04:45:49.650816 1180 tablet_bootstrap.cc:654] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:49.651286 1182 tablet_bootstrap.cc:492] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2: Bootstrap starting.
I20260808 04:45:49.651587 1181 tablet_bootstrap.cc:654] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:49.652323 1182 tablet_bootstrap.cc:654] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:49.652516 1180 tablet_bootstrap.cc:492] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: No bootstrap required, opened a new log
I20260808 04:45:49.652637 1180 ts_tablet_manager.cc:1397] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:49.653213 1180 raft_consensus.cc:348] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:49.653339 1180 raft_consensus.cc:374] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:49.653411 1180 raft_consensus.cc:729] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c1979ae5fb4ef0a97987256b08aa2c, State: Initialized, Role: FOLLOWER
I20260808 04:45:49.653486 1181 tablet_bootstrap.cc:492] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: No bootstrap required, opened a new log
I20260808 04:45:49.653561 1180 consensus_queue.cc:260] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:49.653632 1181 ts_tablet_manager.cc:1397] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:49.653721 1182 tablet_bootstrap.cc:492] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2: No bootstrap required, opened a new log
I20260808 04:45:49.653815 1182 ts_tablet_manager.cc:1397] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:49.653939 1180 ts_tablet_manager.cc:1428] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:49.654063 1181 raft_consensus.cc:348] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:49.654198 1181 raft_consensus.cc:374] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:49.654253 1181 raft_consensus.cc:729] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3aeaed68624f448b39563148adfcbe, State: Initialized, Role: FOLLOWER
I20260808 04:45:49.654333 1182 raft_consensus.cc:348] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:49.654405 1181 consensus_queue.cc:260] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:49.654655 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.654357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:44849)
0808 04:45:49.654529 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.654585 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 04:45:49.654811 1181 ts_tablet_manager.cc:1428] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:49.655489 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.655214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:56441)
0808 04:45:49.655365 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.655418 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 04:45:49.656158 1182 raft_consensus.cc:374] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:49.656263 1182 raft_consensus.cc:729] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e130596245467ab759328b94fd2ed2, State: Initialized, Role: FOLLOWER
I20260808 04:45:49.656442 1182 consensus_queue.cc:260] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:49.656992 1035 heartbeater.cc:503] Master 127.22.2.188:35545 was elected leader, sending a full tablet report...
I20260808 04:45:49.656859 1182 ts_tablet_manager.cc:1428] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260808 04:45:49.657254 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.657003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:36225)
0808 04:45:49.657144 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.657195 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 04:45:49.657469 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.657211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:48513)
0808 04:45:49.657356 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.657406 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 04:45:49.657624 837 heartbeater.cc:503] Master 127.22.2.188:35545 was elected leader, sending a full tablet report...
I20260808 04:45:49.658006 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.657477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:49603)
0808 04:45:49.657903 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.657948 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354}
I20260808 04:45:49.658659 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:49.658409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:41919)
0808 04:45:49.658522 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:49.658586 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 04:45:50.013854 1187 raft_consensus.cc:482] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:50.014014 1187 raft_consensus.cc:504] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:50.015230 1187 leader_election.cc:288] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de3aeaed68624f448b39563148adfcbe (127.22.2.131:41819), b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035)
I20260808 04:45:50.019599 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.015555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:41819 (local address 127.22.2.130:54435)
0808 04:45:50.015672 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.015689 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:50.015800 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:50.016070 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:50.016073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:50.016095 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:50.016253 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:50.016260 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.017267 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.017269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:50.018441 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:50.018452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.019250 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.019253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:50.019341 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:50.019362 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 04:45:50.019423 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 04:45:50.019477 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":18}
I20260808 04:45:50.019704 1174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.015654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:54435 (local address 127.22.2.131:41819)
0808 04:45:50.015782 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:50.015787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:50.015824 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:45:50.015890 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:50.015892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:50.015936 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:50.016013 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:50.016394 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.017155 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.018572 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.019084 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.019257 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:50.019313 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:50.019468 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 04:45:50.019550 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:50.020100 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.015563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45035 (local address 127.22.2.130:37913)
0808 04:45:50.016280 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.016297 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:50.016463 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:50.016775 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:50.016777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:50.016793 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:50.016884 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:50.016889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.017707 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.017710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:50.018767 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:50.018775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.019701 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.019704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:50.019847 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:50.019883 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 04:45:50.019947 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 04:45:50.019996 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":300,"thread_start_us":131,"threads_started":1}
I20260808 04:45:50.020213 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.015654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:37913 (local address 127.22.2.129:45035)
0808 04:45:50.015782 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:50.015786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:50.016389 (+ 603us) server_negotiation.cc:407] Connection header received
0808 04:45:50.016608 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:50.016612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:50.016677 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:50.016757 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:50.016998 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.017590 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.018930 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.019570 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.019697 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:50.019759 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:50.019991 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 04:45:50.020071 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 04:45:50.020617 969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ede6bd4d4c451c9f8ac6dd766be1ac" candidate_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de3aeaed68624f448b39563148adfcbe" is_pre_election: true
I20260808 04:45:50.020807 587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ede6bd4d4c451c9f8ac6dd766be1ac" candidate_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e130596245467ab759328b94fd2ed2" is_pre_election: true
I20260808 04:45:50.020889 969 raft_consensus.cc:2454] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c1979ae5fb4ef0a97987256b08aa2c in term 0.
I20260808 04:45:50.020990 587 raft_consensus.cc:2454] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c1979ae5fb4ef0a97987256b08aa2c in term 0.
I20260808 04:45:50.021433 686 leader_election.cc:302] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b8e130596245467ab759328b94fd2ed2, d8c1979ae5fb4ef0a97987256b08aa2c, de3aeaed68624f448b39563148adfcbe; no voters:
I20260808 04:45:50.021721 1187 raft_consensus.cc:2781] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:50.021812 1187 raft_consensus.cc:482] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:50.021869 1187 raft_consensus.cc:3037] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:50.022967 1187 raft_consensus.cc:504] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:50.023396 1187 leader_election.cc:288] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [CANDIDATE]: Term 1 election: Requested vote from peers de3aeaed68624f448b39563148adfcbe (127.22.2.131:41819), b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035)
I20260808 04:45:50.023752 969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ede6bd4d4c451c9f8ac6dd766be1ac" candidate_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de3aeaed68624f448b39563148adfcbe"
I20260808 04:45:50.023880 969 raft_consensus.cc:3037] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:50.024127 587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ede6bd4d4c451c9f8ac6dd766be1ac" candidate_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e130596245467ab759328b94fd2ed2"
I20260808 04:45:50.024257 587 raft_consensus.cc:3037] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:50.025135 969 raft_consensus.cc:2454] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c1979ae5fb4ef0a97987256b08aa2c in term 1.
I20260808 04:45:50.025485 587 raft_consensus.cc:2454] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c1979ae5fb4ef0a97987256b08aa2c in term 1.
I20260808 04:45:50.025484 686 leader_election.cc:302] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c1979ae5fb4ef0a97987256b08aa2c, de3aeaed68624f448b39563148adfcbe; no voters:
I20260808 04:45:50.025661 1187 raft_consensus.cc:2781] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:50.025870 1187 raft_consensus.cc:686] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 LEADER]: Becoming Leader. State: Replica: d8c1979ae5fb4ef0a97987256b08aa2c, State: Running, Role: LEADER
I20260808 04:45:50.026160 1187 consensus_queue.cc:237] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } }
I20260808 04:45:50.026587 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.026365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:46277)
0808 04:45:50.026466 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.026524 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 04:45:50.028837 391 catalog_manager.cc:5495] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c reported cstate change: term changed from 0 to 1, leader changed from <none> to d8c1979ae5fb4ef0a97987256b08aa2c (127.22.2.130). New cstate: current_term: 1 leader_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c1979ae5fb4ef0a97987256b08aa2c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:50.102227 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.101951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38396)
0808 04:45:50.102074 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.102148 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 04:45:50.102316 1171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.097260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.22.2.130:46229)
0808 04:45:50.097392 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:50.097397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:50.097653 (+ 256us) server_negotiation.cc:407] Connection header received
0808 04:45:50.097817 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:50.097821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:50.097905 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:50.098018 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:50.098347 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.099058 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.100396 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:50.100843 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:50.100975 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:50.101267 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:50.101906 (+ 639us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:50.102038 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:50.102121 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 04:45:50.102178 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 04:45:50.103089 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.102862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.0.0.1:38402)
0808 04:45:50.102973 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.103028 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 04:45:50.105388 587 raft_consensus.cc:1261] T 08ede6bd4d4c451c9f8ac6dd766be1ac P b8e130596245467ab759328b94fd2ed2 [term 1 FOLLOWER]: Refusing update from remote peer d8c1979ae5fb4ef0a97987256b08aa2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:50.105651 969 raft_consensus.cc:1261] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 1 FOLLOWER]: Refusing update from remote peer d8c1979ae5fb4ef0a97987256b08aa2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:50.105856 1187 consensus_queue.cc:1035] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e130596245467ab759328b94fd2ed2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:50.106222 1193 consensus_queue.cc:1035] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "de3aeaed68624f448b39563148adfcbe" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:50.106999 650 heartbeater.cc:503] Master 127.22.2.188:35545 was elected leader, sending a full tablet report...
I20260808 04:45:50.107976 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.107661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:48465)
0808 04:45:50.107841 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.107898 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 04:45:50.108722 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.108395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:43159)
0808 04:45:50.108581 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.108642 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 04:45:50.111469 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.111207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:55223)
0808 04:45:50.111330 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.111388 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 04:45:50.113227 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.112443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.129:50807)
0808 04:45:50.113106 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.113159 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586}
I20260808 04:45:50.113927 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.113581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.131:38413)
0808 04:45:50.113794 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.113859 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260808 04:45:50.115936 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.115694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38987 (local address 127.22.2.130:51931)
0808 04:45:50.115817 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.115872 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:38987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 04:45:50.121858 391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:50.122026 391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:50.126227 391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eaec38a20b3845e0ad056adafb8f6afe: Sending DeleteTablet for 3 replicas of tablet 08ede6bd4d4c451c9f8ac6dd766be1ac
I20260808 04:45:50.127017 943 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:50 UTC) from {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:35274
I20260808 04:45:50.127069 556 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:50 UTC) from {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:53350
I20260808 04:45:50.127085 746 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:50 UTC) from {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:57144
I20260808 04:45:50.127439 1218 tablet_replica.cc:330] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: stopping tablet replica
I20260808 04:45:50.127594 1218 raft_consensus.cc:2229] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:50.127748 1218 raft_consensus.cc:2258] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:50.127761 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 338
I20260808 04:45:50.133461 1218 ts_tablet_manager.cc:1905] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:50.136840 1218 ts_tablet_manager.cc:1918] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:50.137007 1218 log.cc:1196] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/08ede6bd4d4c451c9f8ac6dd766be1ac
I20260808 04:45:50.137423 1218 ts_tablet_manager.cc:1939] T 08ede6bd4d4c451c9f8ac6dd766be1ac P de3aeaed68624f448b39563148adfcbe: Deleting consensus metadata
I20260808 04:45:50.140280 372 catalog_manager.cc:4841] TS de3aeaed68624f448b39563148adfcbe (127.22.2.131:41819): tablet 08ede6bd4d4c451c9f8ac6dd766be1ac (table test-table [id=c5bf001d1c0c4fca831cda65761cf4f5]) successfully deleted
I20260808 04:45:50.140611 1219 tablet_replica.cc:330] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: stopping tablet replica
I20260808 04:45:50.140784 1219 raft_consensus.cc:2229] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:50.141214 1219 raft_consensus.cc:2258] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:50.141883 1219 ts_tablet_manager.cc:1905] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:50.143872 1219 ts_tablet_manager.cc:1918] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:50.143970 1219 log.cc:1196] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/08ede6bd4d4c451c9f8ac6dd766be1ac
I20260808 04:45:50.144266 1219 ts_tablet_manager.cc:1939] T 08ede6bd4d4c451c9f8ac6dd766be1ac P d8c1979ae5fb4ef0a97987256b08aa2c: Deleting consensus metadata
I20260808 04:45:50.145011 373 catalog_manager.cc:4841] TS d8c1979ae5fb4ef0a97987256b08aa2c (127.22.2.130:46229): tablet 08ede6bd4d4c451c9f8ac6dd766be1ac (table test-table [id=c5bf001d1c0c4fca831cda65761cf4f5]) successfully deleted
I20260808 04:45:50.149586 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 660
I20260808 04:45:50.152267 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:50.149713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45035 (local address 127.0.0.1:53360)
0808 04:45:50.152092 (+ 2379us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:50.152155 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2268}
W20260808 04:45:50.154488 373 catalog_manager.cc:4584] TS b8e130596245467ab759328b94fd2ed2 (127.22.2.129:45035): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08ede6bd4d4c451c9f8ac6dd766be1ac: Network error: Client connection negotiation failed: client connection to 127.22.2.129:45035: connect: Connection refused (error 111)
I20260808 04:45:50.160277 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 849
I20260808 04:45:50.169700 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 32410
I20260808 04:45:50.181360 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 32552
2026-08-08T04:45:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7863 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:50 dist-test-slave-04bb krb5kdc[1224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:50 dist-test-slave-04bb krb5kdc[1224](info): set up 2 sockets
Aug 08 04:45:50 dist-test-slave-04bb krb5kdc[1224](info): commencing operation
krb5kdc: starting...
W20260808 04:45:51.440043 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:51 dist-test-slave-04bb krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.190.keytab.
Entry for principal oryx/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.190.keytab.
I20260808 04:45:51.567783 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42255
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.190.keytab
--principal=oryx/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:51.678674 1245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:51.678910 1245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:51.678948 1245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:51.682842 1245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:51.682894 1245 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 04:45:51.682911 1245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:51.682929 1245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:51.682945 1245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:51.682960 1245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:51.688089 1245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42255
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.190.keytab
--principal=oryx/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:51.689093 1245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:51.690102 1245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:51.699069 1252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:51.699142 1250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:51.699180 1254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:51.699220 1245 server_base.cc:1034] running on GCE node
I20260808 04:45:51.699976 1245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:51.701138 1245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:51.702310 1245 hybrid_clock.cc:648] HybridClock initialized: now 1786164351702277 us; error 41 us; skew 500 ppm
Aug 08 04:45:51 dist-test-slave-04bb krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164351, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:51.705104 1245 init.cc:377] Logged in from keytab as oryx/127.22.2.190@KRBTEST.COM (short username oryx)
I20260808 04:45:51.706038 1245 webserver.cc:460] Webserver started at http://127.22.2.190:43849/ using document root <none> and password file <none>
I20260808 04:45:51.706609 1245 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:51.706668 1245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:51.706890 1245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:51.708982 1245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "5edbecaf380b45d59eb35bbaeab6f3b6"
format_stamp: "Formatted at 2026-08-08 04:45:51 on dist-test-slave-04bb"
server_key: "53667d2bc7423bffaf8ac11c9d226130"
server_key_iv: "83e786b897b9b8ce24fca9fe7e4588e7"
server_key_version: "encryptionkey@0"
I20260808 04:45:51.709553 1245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "5edbecaf380b45d59eb35bbaeab6f3b6"
format_stamp: "Formatted at 2026-08-08 04:45:51 on dist-test-slave-04bb"
server_key: "53667d2bc7423bffaf8ac11c9d226130"
server_key_iv: "83e786b897b9b8ce24fca9fe7e4588e7"
server_key_version: "encryptionkey@0"
I20260808 04:45:51.713193 1245 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:45:51.715543 1264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:51.716778 1245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 04:45:51.716913 1245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "5edbecaf380b45d59eb35bbaeab6f3b6"
format_stamp: "Formatted at 2026-08-08 04:45:51 on dist-test-slave-04bb"
server_key: "53667d2bc7423bffaf8ac11c9d226130"
server_key_iv: "83e786b897b9b8ce24fca9fe7e4588e7"
server_key_version: "encryptionkey@0"
I20260808 04:45:51.717020 1245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:51.729946 1245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:51.732949 1245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:51.733141 1245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:51.741091 1245 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42255
I20260808 04:45:51.741120 1358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42255 every 8 connection(s)
I20260808 04:45:51.742012 1245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:51.743600 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 1245
I20260808 04:45:51.743745 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:51.744064 22538 external_mini_cluster.cc:1428] Setting key 794c5701ed6811d585a0eb36b7084b1a
I20260808 04:45:51.745121 1362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:51.751765 1362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: Bootstrap starting.
Aug 08 04:45:51 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:51.754314 1362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:51.755343 1362 log.cc:824] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:51.757730 1362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: No bootstrap required, opened a new log
I20260808 04:45:51.759755 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:51.745577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45888 (local address 127.22.2.190:42255)
0808 04:45:51.745996 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:51.746015 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:51.746059 (+ 44us) server_negotiation.cc:407] Connection header received
0808 04:45:51.746673 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:51.746688 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:51.747007 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:51.747272 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:51.748242 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:51.749323 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:51.750545 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:51.751000 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:51.751116 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:51.753924 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:51.753930 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:51.753943 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:51.753969 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:51.756713 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:51.757146 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:51.757152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:51.757158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:51.757301 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:51.757570 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:51.757573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:51.757575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:51.758071 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:51.758272 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:51.758528 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 04:45:51.758819 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":144,"threads_started":1}
I20260808 04:45:51.761518 1362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edbecaf380b45d59eb35bbaeab6f3b6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42255 } }
I20260808 04:45:51.761793 1362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:51.761866 1362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edbecaf380b45d59eb35bbaeab6f3b6, State: Initialized, Role: FOLLOWER
I20260808 04:45:51.762336 1362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edbecaf380b45d59eb35bbaeab6f3b6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42255 } }
I20260808 04:45:51.762468 1362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:51.762558 1362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:51.762689 1362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:51.763549 1362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edbecaf380b45d59eb35bbaeab6f3b6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42255 } }
I20260808 04:45:51.763900 1362 leader_election.cc:302] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5edbecaf380b45d59eb35bbaeab6f3b6; no voters:
I20260808 04:45:51.764218 1362 leader_election.cc:288] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:51.764374 1367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:51.764636 1367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [term 1 LEADER]: Becoming Leader. State: Replica: 5edbecaf380b45d59eb35bbaeab6f3b6, State: Running, Role: LEADER
I20260808 04:45:51.764935 1367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edbecaf380b45d59eb35bbaeab6f3b6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42255 } }
I20260808 04:45:51.765725 1362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:51.766600 1371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5edbecaf380b45d59eb35bbaeab6f3b6. Latest consensus state: current_term: 1 leader_uuid: "5edbecaf380b45d59eb35bbaeab6f3b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edbecaf380b45d59eb35bbaeab6f3b6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42255 } } }
I20260808 04:45:51.767007 1371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:51.766901 1372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5edbecaf380b45d59eb35bbaeab6f3b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edbecaf380b45d59eb35bbaeab6f3b6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42255 } } }
I20260808 04:45:51.767338 1372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6 [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:51.767409 1379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:51.770665 1379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:51.776329 1379 catalog_manager.cc:1347] Generated new cluster ID: a38cdca3e52e4b4bb00820bfa387d1ff
I20260808 04:45:51.776427 1379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:51.785599 1379 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:51.786767 1379 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:51.795323 1379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: Generated new TSK 0
I20260808 04:45:51.796105 1379 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
I20260808 04:45:51.833236 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42255
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:51.948194 1397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:51.948446 1397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:51.948491 1397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:51.952538 1397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:51.952592 1397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:51.952630 1397 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:51.952662 1397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:51.958293 1397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:51.959309 1397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:51.960254 1397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:51.970299 1404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:51.970367 1402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:51.970360 1406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:51.970654 1397 server_base.cc:1034] running on GCE node
I20260808 04:45:51.971046 1397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:51.971704 1397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:51.972882 1397 hybrid_clock.cc:648] HybridClock initialized: now 1786164351972854 us; error 39 us; skew 500 ppm
Aug 08 04:45:51 dist-test-slave-04bb krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164351, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:51.975756 1397 init.cc:377] Logged in from keytab as oryx/127.22.2.129@KRBTEST.COM (short username oryx)
I20260808 04:45:51.976617 1397 webserver.cc:460] Webserver started at http://127.22.2.129:43789/ using document root <none> and password file <none>
I20260808 04:45:51.977137 1397 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:51.977195 1397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:51.977368 1397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:51.979489 1397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "c479eb479bde4c1fb407d952b1bf7987"
format_stamp: "Formatted at 2026-08-08 04:45:51 on dist-test-slave-04bb"
server_key: "8699ed1fd517244164ed4caaea78d30e"
server_key_iv: "8b246fe1876973361d4a6bb4e9b154d8"
server_key_version: "encryptionkey@0"
I20260808 04:45:51.980001 1397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "c479eb479bde4c1fb407d952b1bf7987"
format_stamp: "Formatted at 2026-08-08 04:45:51 on dist-test-slave-04bb"
server_key: "8699ed1fd517244164ed4caaea78d30e"
server_key_iv: "8b246fe1876973361d4a6bb4e9b154d8"
server_key_version: "encryptionkey@0"
I20260808 04:45:51.983610 1397 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:45:51.985993 1416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:51.987198 1397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 04:45:51.987329 1397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "c479eb479bde4c1fb407d952b1bf7987"
format_stamp: "Formatted at 2026-08-08 04:45:51 on dist-test-slave-04bb"
server_key: "8699ed1fd517244164ed4caaea78d30e"
server_key_iv: "8b246fe1876973361d4a6bb4e9b154d8"
server_key_version: "encryptionkey@0"
I20260808 04:45:51.987433 1397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:52.007666 1397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:52.011152 1397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:52.011375 1397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:52.013000 1397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:52.013062 1397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:52.013116 1397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:52.013181 1397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164351, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:52.025846 1397 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:43707
I20260808 04:45:52.025946 1576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:43707 every 8 connection(s)
I20260808 04:45:52.027061 1397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:52.028976 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.013940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:57759 (local address 127.22.2.190:42255)
0808 04:45:52.014130 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.014139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.014996 (+ 857us) server_negotiation.cc:407] Connection header received
0808 04:45:52.017152 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.017157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.017231 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.017346 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:52.018667 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.019269 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.020866 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.021246 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.021390 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.024758 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:52.024762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:52.024765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:52.024795 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:52.026690 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.027240 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.027244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.027246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.027341 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.027776 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.027779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.027781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.028031 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:52.028145 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.028740 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.028854 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 04:45:52.029770 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.014329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42255 (local address 127.22.2.129:57759)
0808 04:45:52.014807 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.014862 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.016906 (+ 2044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.017530 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.017537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.017985 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:52.018496 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.018511 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.019430 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.019439 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.020724 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.020736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.021453 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.021463 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.021726 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.022467 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:52.022492 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:52.024584 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:52.026822 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.026827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.026838 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.027111 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.027439 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.027444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.027446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.027650 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.028158 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:52.028164 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:52.028437 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.028672 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.028931 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":138,"threads_started":1}
I20260808 04:45:52.030079 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 1397
I20260808 04:45:52.030236 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:52.030612 22538 external_mini_cluster.cc:1428] Setting key acb3c735ff3d0e6b4ec76680c052f924
I20260808 04:45:52.031488 1577 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42255
I20260808 04:45:52.031642 1577 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:52.032176 1577 heartbeater.cc:511] Master 127.22.2.190:42255 requested a full tablet report, sending...
I20260808 04:45:52.033841 1290 ts_manager.cc:194] Registered new tserver with Master: c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707)
I20260808 04:45:52.035441 1290 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:57759
WARNING: no policy specified for oryx/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:45:52.044651 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.036125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:60199 (local address 127.22.2.190:42255)
0808 04:45:52.036241 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.036245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.036265 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:45:52.036294 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.036296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.036329 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.036427 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:52.037016 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.037802 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.039153 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.039530 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.039659 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.041150 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:52.041155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:52.041158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:52.041192 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:52.042698 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.043194 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.043199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.043201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.043295 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.043615 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.043619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.043621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.043854 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:52.043977 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.044314 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.044439 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:45:52.044670 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.033817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42255 (local address 127.22.2.129:60199)
0808 04:45:52.034141 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.034156 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.034249 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.036447 (+ 2198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.036450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.036703 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:52.036867 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.036878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.037964 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.037971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.038992 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.039006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.039809 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.039816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.039950 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.040634 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:52.040648 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:52.040975 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:52.042851 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.042854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.042857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.043069 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.043400 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.043404 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.043406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.043505 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.043967 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:52.043970 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:52.044120 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.044253 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.044403 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
I20260808 04:45:52.063365 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42255
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:52.182034 1592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:52.182322 1592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:52.182413 1592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:52.186398 1592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:52.186481 1592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:52.186553 1592 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:52.186630 1592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:52.191767 1592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:52.192870 1592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:52.193919 1592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:52.203609 1599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:52.203595 1600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:52.203747 1592 server_base.cc:1034] running on GCE node
W20260808 04:45:52.203616 1605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:52.204299 1592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:52.204999 1592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:52.206233 1592 hybrid_clock.cc:648] HybridClock initialized: now 1786164352206196 us; error 46 us; skew 500 ppm
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164352, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:52.209499 1592 init.cc:377] Logged in from keytab as oryx/127.22.2.130@KRBTEST.COM (short username oryx)
I20260808 04:45:52.210427 1592 webserver.cc:460] Webserver started at http://127.22.2.130:38633/ using document root <none> and password file <none>
I20260808 04:45:52.210991 1592 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:52.211053 1592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:52.211231 1592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:52.213508 1592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "c91961de15e34721bb99afeb90e13dc9"
format_stamp: "Formatted at 2026-08-08 04:45:52 on dist-test-slave-04bb"
server_key: "3e9078139132c1d5f046e52acfafafe4"
server_key_iv: "9f94d85d08cce5fd302517a4d6f7685e"
server_key_version: "encryptionkey@0"
I20260808 04:45:52.214090 1592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "c91961de15e34721bb99afeb90e13dc9"
format_stamp: "Formatted at 2026-08-08 04:45:52 on dist-test-slave-04bb"
server_key: "3e9078139132c1d5f046e52acfafafe4"
server_key_iv: "9f94d85d08cce5fd302517a4d6f7685e"
server_key_version: "encryptionkey@0"
I20260808 04:45:52.218135 1592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:45:52.220646 1615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:52.222036 1592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:45:52.222174 1592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "c91961de15e34721bb99afeb90e13dc9"
format_stamp: "Formatted at 2026-08-08 04:45:52 on dist-test-slave-04bb"
server_key: "3e9078139132c1d5f046e52acfafafe4"
server_key_iv: "9f94d85d08cce5fd302517a4d6f7685e"
server_key_version: "encryptionkey@0"
I20260808 04:45:52.222294 1592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:52.251617 1592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:52.255239 1592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:52.255473 1592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:52.257174 1592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:52.257256 1592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:52.257371 1592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:52.257416 1592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164352, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:52.269855 1592 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41877
I20260808 04:45:52.269906 1771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41877 every 8 connection(s)
I20260808 04:45:52.271097 1592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:52.272850 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.257881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:45595 (local address 127.22.2.190:42255)
0808 04:45:52.258012 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.258015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.259221 (+ 1206us) server_negotiation.cc:407] Connection header received
0808 04:45:52.260327 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.260332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.260430 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.260566 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:52.261887 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.262432 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.264357 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.264746 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.264879 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.268484 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:52.268491 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:52.268495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:52.268595 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:52.270491 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.271018 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.271050 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.271055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.271173 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.271583 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.271587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.271589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.271841 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:52.271948 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.272604 (+ 656us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.272728 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 04:45:52.273613 1627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.258431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42255 (local address 127.22.2.130:45595)
0808 04:45:52.259037 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.259103 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.260055 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.260765 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.260773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.261170 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:52.261642 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.261656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.262619 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.262623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.264179 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.264195 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.264946 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.264951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.265187 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.265920 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:52.265948 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:52.268259 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:52.270633 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.270638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.270650 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.270903 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.271327 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.271330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.271332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.271476 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.272001 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:52.272034 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:52.272316 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.272474 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.272778 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":164,"threads_started":1}
I20260808 04:45:52.274856 1774 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42255
I20260808 04:45:52.275009 1774 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:52.275524 1774 heartbeater.cc:511] Master 127.22.2.190:42255 requested a full tablet report, sending...
I20260808 04:45:52.276635 1291 ts_manager.cc:194] Registered new tserver with Master: c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877)
I20260808 04:45:52.277362 1291 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:45595
I20260808 04:45:52.280342 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 1592
I20260808 04:45:52.280471 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:52.280890 22538 external_mini_cluster.cc:1428] Setting key 14ba5239bb18ebffda6ccf00e58585ce
I20260808 04:45:52.284943 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.277087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:57175 (local address 127.22.2.190:42255)
0808 04:45:52.277215 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.277221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.277455 (+ 234us) server_negotiation.cc:407] Connection header received
0808 04:45:52.277583 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.277587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.277637 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.277758 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:52.278240 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.278986 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.279945 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.280217 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.280343 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.281526 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:52.281531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:52.281534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:52.281562 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:52.283038 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.283501 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.283506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.283507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.283602 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.283912 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.283917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.283919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.284181 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:52.284312 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.284729 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.284838 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:45:52.285123 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.277011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42255 (local address 127.22.2.130:57175)
0808 04:45:52.277362 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.277373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.277477 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.277777 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.277781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.277983 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:52.278086 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.278092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.279216 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.279254 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.279827 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.279836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.280342 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.280345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.280457 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.281047 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:52.281060 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:52.281375 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:52.283174 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.283178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.283180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.283389 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.283706 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.283711 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.283713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.283807 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.284372 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:52.284377 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:52.284502 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.284673 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.284850 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for oryx/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
I20260808 04:45:52.311975 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42255
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:52.426926 1786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:52.427186 1786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:52.427255 1786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:52.431303 1786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:52.431380 1786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:52.431447 1786 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:45:52.431524 1786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:52.436627 1786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:52.437631 1786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:52.438630 1786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:52.448395 1792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:52.448490 1786 server_base.cc:1034] running on GCE node
W20260808 04:45:52.448434 1796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:52.448441 1793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:52.448987 1786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:52.449630 1786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:52.450815 1786 hybrid_clock.cc:648] HybridClock initialized: now 1786164352450805 us; error 32 us; skew 500 ppm
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164352, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:52.453727 1786 init.cc:377] Logged in from keytab as oryx/127.22.2.131@KRBTEST.COM (short username oryx)
I20260808 04:45:52.454602 1786 webserver.cc:460] Webserver started at http://127.22.2.131:46641/ using document root <none> and password file <none>
I20260808 04:45:52.455153 1786 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:52.455241 1786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:52.455469 1786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:52.457525 1786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "aba0e2ec56434d59a761b0ec93b95792"
format_stamp: "Formatted at 2026-08-08 04:45:52 on dist-test-slave-04bb"
server_key: "38d00e0b06d133c6f489b45102131d40"
server_key_iv: "5a442ac5537b74d0a5e2de61c3dd1f92"
server_key_version: "encryptionkey@0"
I20260808 04:45:52.458128 1786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "aba0e2ec56434d59a761b0ec93b95792"
format_stamp: "Formatted at 2026-08-08 04:45:52 on dist-test-slave-04bb"
server_key: "38d00e0b06d133c6f489b45102131d40"
server_key_iv: "5a442ac5537b74d0a5e2de61c3dd1f92"
server_key_version: "encryptionkey@0"
I20260808 04:45:52.461776 1786 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:45:52.464134 1805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:52.465430 1786 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:52.465562 1786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "aba0e2ec56434d59a761b0ec93b95792"
format_stamp: "Formatted at 2026-08-08 04:45:52 on dist-test-slave-04bb"
server_key: "38d00e0b06d133c6f489b45102131d40"
server_key_iv: "5a442ac5537b74d0a5e2de61c3dd1f92"
server_key_version: "encryptionkey@0"
I20260808 04:45:52.465683 1786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:52.482852 1786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:52.486431 1786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:52.486644 1786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:52.488131 1786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:52.488205 1786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:52.488279 1786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:52.488322 1786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164352, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:52.500090 1786 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:42025
I20260808 04:45:52.500144 1973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:42025 every 8 connection(s)
I20260808 04:45:52.501287 1786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:52.502839 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.489132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:33101 (local address 127.22.2.190:42255)
0808 04:45:52.489319 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.489324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.490301 (+ 977us) server_negotiation.cc:407] Connection header received
0808 04:45:52.491667 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.491672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.491739 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.491865 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:52.493412 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.493955 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.495465 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.495701 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.495831 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.498676 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:52.498683 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:52.498689 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:52.498724 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:52.500404 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.501087 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.501099 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.501101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.501203 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.501646 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.501650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.501652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.501919 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:52.502031 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.502614 (+ 583us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.502716 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 04:45:52.503629 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.489539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42255 (local address 127.22.2.131:33101)
0808 04:45:52.490047 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.490171 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.491417 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.492116 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.492125 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.492598 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:52.493239 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.493259 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.494088 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.494091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.495336 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.495346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.495889 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.495894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.496113 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.496820 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:52.496837 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:52.498488 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:52.500520 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.500578 (+ 58us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.500593 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.500979 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.501325 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.501327 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.501328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.501514 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.502102 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:52.502110 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:52.502366 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.502520 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.502761 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260808 04:45:52.504889 1974 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42255
I20260808 04:45:52.505051 1974 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:52.505568 1974 heartbeater.cc:511] Master 127.22.2.190:42255 requested a full tablet report, sending...
I20260808 04:45:52.506683 1290 ts_manager.cc:194] Registered new tserver with Master: aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:52.507388 1290 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:33101
I20260808 04:45:52.509050 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 1786
I20260808 04:45:52.509219 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:52.509547 22538 external_mini_cluster.cc:1428] Setting key 12fa24212cfb19ecdea39e7b2839376a
I20260808 04:45:52.511684 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:45:52.514690 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.507138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:45473 (local address 127.22.2.190:42255)
0808 04:45:52.507269 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.507275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.507505 (+ 230us) server_negotiation.cc:407] Connection header received
0808 04:45:52.507680 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.507683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.507733 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.507813 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:52.508330 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.508955 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.509941 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.510214 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.510329 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.511549 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:52.511553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:52.511556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:52.511586 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:52.513056 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.513515 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.513518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.513520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.513600 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.513884 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.513888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.513890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.514061 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:52.514181 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.514424 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.514531 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:45:52.514724 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.507045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42255 (local address 127.22.2.131:45473)
0808 04:45:52.507409 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.507421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.507575 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.507856 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.507858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.508071 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:52.508203 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.508211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.509102 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.509105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.509836 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.509846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.510349 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.510353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.510465 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.511064 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:52.511076 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:52.511405 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:52.513208 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.513211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.513214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.513409 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.513700 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:52.513704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:52.513706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:52.513792 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:52.514161 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:52.514164 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:52.514256 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.514369 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.514517 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":113,"threads_started":1}
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 04:45:52.537029 1365 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:52.531052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.22.2.190:42255)
0808 04:45:52.531179 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.531183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.531196 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:45:52.531237 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.531240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.531282 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.531359 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:52.532085 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.532670 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.533492 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.533698 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.533791 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.536864 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45900: BlockingRecv error: recv got EOF from 127.0.0.1:45900 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 04:45:52 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:52.546715 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.538747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.22.2.190:42255)
0808 04:45:52.538888 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.538891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.538947 (+ 56us) server_negotiation.cc:407] Connection header received
0808 04:45:52.539034 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.539037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.539072 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.539137 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:52.539687 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.540145 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.540962 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.541158 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.541236 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.543497 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:52.543500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:52.543502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:52.543524 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:52.544941 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.545425 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.545427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.545428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.545505 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:52.545761 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:52.545763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:52.545764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:52.545928 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:52.546030 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.546399 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.546546 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:45:52.549687 1290 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:52.552713 1290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:52.567768 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.562037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40186 (local address 127.22.2.131:42025)
0808 04:45:52.562359 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.562370 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.562399 (+ 29us) server_negotiation.cc:407] Connection header received
0808 04:45:52.562461 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.562464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.562622 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.562880 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:52.563136 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.564247 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.566046 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.566909 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.567192 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.567380 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.567508 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.567607 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":121,"threads_started":1}
I20260808 04:45:52.567764 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.561576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41877 (local address 127.0.0.1:35024)
0808 04:45:52.562175 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.562190 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.562283 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.563004 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.563007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.563024 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:52.563117 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.563123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.564198 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.564204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.565875 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.565888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.567059 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.567063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.567255 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.567321 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.567432 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.567601 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":182,"thread_start_us":93,"threads_started":1}
I20260808 04:45:52.567996 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.562138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.22.2.130:41877)
0808 04:45:52.562549 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.562565 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.562589 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:45:52.562666 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.562670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.562845 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.563255 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:52.563369 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.564073 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.566033 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.566910 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.567523 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.567685 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.567735 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.567862 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":137,"threads_started":1}
I20260808 04:45:52.568436 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.561020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:42025 (local address 127.0.0.1:40186)
0808 04:45:52.561573 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.561587 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.561672 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.562766 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.562768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.562800 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:52.563018 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.563025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.564389 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.564393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.565916 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.565927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.567059 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.567063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.567320 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.567337 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.567460 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.568338 (+ 878us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1060,"thread_start_us":91,"threads_started":1}
I20260808 04:45:52.568828 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.561041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60812 (local address 127.22.2.129:43707)
0808 04:45:52.561431 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.561440 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.561734 (+ 294us) server_negotiation.cc:407] Connection header received
0808 04:45:52.561839 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.561845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.562507 (+ 662us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.562677 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:52.563055 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.564448 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.566046 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.566848 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.566980 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.567145 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.568560 (+ 1415us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.568669 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":96,"threads_started":1}
I20260808 04:45:52.569078 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.560915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43707 (local address 127.0.0.1:60812)
0808 04:45:52.561397 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.561446 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.561577 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.562677 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.562686 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.562714 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:52.562923 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.562931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.564593 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.564597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.565897 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.565913 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.567213 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.567217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.567306 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.567331 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.567431 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.568454 (+ 1023us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":37,"thread_start_us":58,"threads_started":1}
I20260808 04:45:52.571355 1875 tablet_service.cc:1467] Processing CreateTablet for tablet 512b475ecd3f432c89f6f229e7f62410 (DEFAULT_TABLE table=test-table [id=1e0a928c471b47268007978b16bcde59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:52.571444 1679 tablet_service.cc:1467] Processing CreateTablet for tablet 512b475ecd3f432c89f6f229e7f62410 (DEFAULT_TABLE table=test-table [id=1e0a928c471b47268007978b16bcde59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:52.571482 1491 tablet_service.cc:1467] Processing CreateTablet for tablet 512b475ecd3f432c89f6f229e7f62410 (DEFAULT_TABLE table=test-table [id=1e0a928c471b47268007978b16bcde59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:52.572491 1875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512b475ecd3f432c89f6f229e7f62410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:52.572499 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512b475ecd3f432c89f6f229e7f62410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:52.572500 1491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512b475ecd3f432c89f6f229e7f62410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:52.579058 2016 tablet_bootstrap.cc:492] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Bootstrap starting.
I20260808 04:45:52.580741 2018 tablet_bootstrap.cc:492] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Bootstrap starting.
I20260808 04:45:52.580823 2017 tablet_bootstrap.cc:492] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Bootstrap starting.
I20260808 04:45:52.581317 2016 tablet_bootstrap.cc:654] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:52.582234 2016 log.cc:824] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:52.582916 2018 tablet_bootstrap.cc:654] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:52.582916 2017 tablet_bootstrap.cc:654] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:52.583761 2017 log.cc:824] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:52.583761 2018 log.cc:824] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:52.584048 2016 tablet_bootstrap.cc:492] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: No bootstrap required, opened a new log
I20260808 04:45:52.584254 2016 ts_tablet_manager.cc:1397] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 04:45:52.585742 2017 tablet_bootstrap.cc:492] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: No bootstrap required, opened a new log
I20260808 04:45:52.585930 2017 ts_tablet_manager.cc:1397] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:45:52.586025 2018 tablet_bootstrap.cc:492] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: No bootstrap required, opened a new log
I20260808 04:45:52.586192 2018 ts_tablet_manager.cc:1397] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:45:52.587500 2016 raft_consensus.cc:348] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.587729 2016 raft_consensus.cc:374] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:52.587795 2016 raft_consensus.cc:729] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba0e2ec56434d59a761b0ec93b95792, State: Initialized, Role: FOLLOWER
I20260808 04:45:52.588258 2016 consensus_queue.cc:260] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.589313 2016 ts_tablet_manager.cc:1428] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260808 04:45:52.589687 1974 heartbeater.cc:503] Master 127.22.2.190:42255 was elected leader, sending a full tablet report...
I20260808 04:45:52.590057 2018 raft_consensus.cc:348] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.590325 2018 raft_consensus.cc:374] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:52.590394 2018 raft_consensus.cc:729] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91961de15e34721bb99afeb90e13dc9, State: Initialized, Role: FOLLOWER
I20260808 04:45:52.590348 2017 raft_consensus.cc:348] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.590585 2017 raft_consensus.cc:374] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:52.590637 2017 raft_consensus.cc:729] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c479eb479bde4c1fb407d952b1bf7987, State: Initialized, Role: FOLLOWER
I20260808 04:45:52.590871 2018 consensus_queue.cc:260] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.591045 2017 consensus_queue.cc:260] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.591575 1774 heartbeater.cc:503] Master 127.22.2.190:42255 was elected leader, sending a full tablet report...
I20260808 04:45:52.591910 2017 ts_tablet_manager.cc:1428] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260808 04:45:52.592258 1577 heartbeater.cc:503] Master 127.22.2.190:42255 was elected leader, sending a full tablet report...
I20260808 04:45:52.592887 2018 ts_tablet_manager.cc:1428] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260808 04:45:52.752319 1977 tablet.cc:2369] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:52.767454 2024 raft_consensus.cc:482] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:52.767694 2024 raft_consensus.cc:504] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.769126 2024 leader_election.cc:288] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
W20260808 04:45:52.772274 1776 tablet.cc:2369] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:52.773344 1627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.769391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43707 (local address 127.22.2.130:56423)
0808 04:45:52.769721 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.769735 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.769855 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.770128 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.770131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.770146 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:52.770267 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.770274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.771131 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.771134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.772048 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.772057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.772884 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.772888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.773004 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.773022 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.773113 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.773187 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":22}
I20260808 04:45:52.773330 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.769474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:56423 (local address 127.22.2.129:43707)
0808 04:45:52.769617 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.769622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.769802 (+ 180us) server_negotiation.cc:407] Connection header received
0808 04:45:52.769976 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.769984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.770030 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.770118 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:52.770387 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.771024 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.772173 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.772762 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.772893 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.772955 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.773128 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.773195 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 04:45:52.773574 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.769554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:42307 (local address 127.22.2.131:42025)
0808 04:45:52.769693 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.769698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.770222 (+ 524us) server_negotiation.cc:407] Connection header received
0808 04:45:52.770297 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.770303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.770365 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.770461 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:52.770730 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.771311 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.772500 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.773037 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.773170 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.773219 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.773397 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.773466 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 04:45:52.773602 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.769423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:42025 (local address 127.22.2.130:42307)
0808 04:45:52.769820 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:52.769837 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:52.769950 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:52.770475 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:52.770478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:52.770497 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:52.770620 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.770626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.771419 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.771422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.772324 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:52.772335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.773163 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.773166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:52.773265 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.773286 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 04:45:52.773382 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 04:45:52.773448 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":40,"thread_start_us":98,"threads_started":1}
I20260808 04:45:52.774298 1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512b475ecd3f432c89f6f229e7f62410" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c479eb479bde4c1fb407d952b1bf7987" is_pre_election: true
I20260808 04:45:52.774623 1516 raft_consensus.cc:2454] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 0.
I20260808 04:45:52.775164 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512b475ecd3f432c89f6f229e7f62410" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792" is_pre_election: true
I20260808 04:45:52.775460 1903 raft_consensus.cc:2454] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 0.
I20260808 04:45:52.775599 1618 leader_election.cc:302] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c479eb479bde4c1fb407d952b1bf7987, c91961de15e34721bb99afeb90e13dc9; no voters:
I20260808 04:45:52.775907 2024 raft_consensus.cc:2781] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:52.776049 2024 raft_consensus.cc:482] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:52.776113 2024 raft_consensus.cc:3037] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:52.777249 2024 raft_consensus.cc:504] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.777601 2024 leader_election.cc:288] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 election: Requested vote from peers c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:52.777964 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512b475ecd3f432c89f6f229e7f62410" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792"
I20260808 04:45:52.778065 1903 raft_consensus.cc:3037] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Advancing to term 1
W20260808 04:45:52.778633 1581 tablet.cc:2369] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:45:52.779515 1903 raft_consensus.cc:2454] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 1.
I20260808 04:45:52.779824 1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512b475ecd3f432c89f6f229e7f62410" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c479eb479bde4c1fb407d952b1bf7987"
I20260808 04:45:52.779908 1620 leader_election.cc:302] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aba0e2ec56434d59a761b0ec93b95792, c91961de15e34721bb99afeb90e13dc9; no voters:
I20260808 04:45:52.780014 1516 raft_consensus.cc:3037] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:52.780073 2024 raft_consensus.cc:2781] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:52.780326 2024 raft_consensus.cc:686] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 1 LEADER]: Becoming Leader. State: Replica: c91961de15e34721bb99afeb90e13dc9, State: Running, Role: LEADER
I20260808 04:45:52.780691 2024 consensus_queue.cc:237] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } }
I20260808 04:45:52.781268 1516 raft_consensus.cc:2454] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 1.
I20260808 04:45:52.784297 1290 catalog_manager.cc:5495] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c91961de15e34721bb99afeb90e13dc9 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } health_report { overall_health: HEALTHY } } }
I20260808 04:45:52.799122 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:52.795619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35026 (local address 127.22.2.130:41877)
0808 04:45:52.795745 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:52.795749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:52.795763 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:45:52.795897 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:52.795900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:52.795953 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:52.796038 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:52.796242 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.796825 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.797755 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:52.797986 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:52.798076 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:52.798317 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:52.798730 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:52.798830 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:52.798889 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 04:45:52.798938 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:45:52.804965 1903 raft_consensus.cc:1261] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Refusing update from remote peer c91961de15e34721bb99afeb90e13dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:52.804966 1516 raft_consensus.cc:1261] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Refusing update from remote peer c91961de15e34721bb99afeb90e13dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:52.805691 2024 consensus_queue.cc:1035] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:52.805905 2027 consensus_queue.cc:1035] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:52.813918 2036 mvcc.cc:205] Tried to move back new op lower bound from 7316129189081722880 to 7316129188992643072. Current Snapshot: MvccSnapshot[applied={T|T < 7316129189081722880}]
I20260808 04:45:52.815155 2035 mvcc.cc:205] Tried to move back new op lower bound from 7316129189081722880 to 7316129188992643072. Current Snapshot: MvccSnapshot[applied={T|T < 7316129189081722880}]
I20260808 04:45:52.822629 1291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:52.823004 1291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:52.825529 1291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: Sending DeleteTablet for 3 replicas of tablet 512b475ecd3f432c89f6f229e7f62410
I20260808 04:45:52.826426 1679 tablet_service.cc:1514] Processing DeleteTablet for tablet 512b475ecd3f432c89f6f229e7f62410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:52 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:35024
I20260808 04:45:52.826422 1491 tablet_service.cc:1514] Processing DeleteTablet for tablet 512b475ecd3f432c89f6f229e7f62410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:52 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60812
I20260808 04:45:52.826961 2048 tablet_replica.cc:330] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: stopping tablet replica
I20260808 04:45:52.827129 1875 tablet_service.cc:1514] Processing DeleteTablet for tablet 512b475ecd3f432c89f6f229e7f62410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:52 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40186
I20260808 04:45:52.827350 2048 raft_consensus.cc:2229] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:52.827696 2048 raft_consensus.cc:2258] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:52.827891 2049 tablet_replica.cc:330] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: stopping tablet replica
I20260808 04:45:52.827958 1291 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 04:45:52.828298 2047 tablet_replica.cc:330] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: stopping tablet replica
I20260808 04:45:52.828341 2049 raft_consensus.cc:2229] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 04:45:52.828405 1291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:52.828672 2047 raft_consensus.cc:2229] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:52.828824 2049 raft_consensus.cc:2258] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:52.829159 2047 raft_consensus.cc:2258] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:52.830230 2048 ts_tablet_manager.cc:1905] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:52.830579 2047 ts_tablet_manager.cc:1905] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:52.831643 2049 ts_tablet_manager.cc:1905] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:52.833256 2048 ts_tablet_manager.cc:1918] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:52.833333 2047 ts_tablet_manager.cc:1918] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:52.833376 2048 log.cc:1196] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/512b475ecd3f432c89f6f229e7f62410
I20260808 04:45:52.833413 2047 log.cc:1196] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/512b475ecd3f432c89f6f229e7f62410
I20260808 04:45:52.833715 2048 ts_tablet_manager.cc:1939] T 512b475ecd3f432c89f6f229e7f62410 P c479eb479bde4c1fb407d952b1bf7987: Deleting consensus metadata
I20260808 04:45:52.833715 2047 ts_tablet_manager.cc:1939] T 512b475ecd3f432c89f6f229e7f62410 P c91961de15e34721bb99afeb90e13dc9: Deleting consensus metadata
I20260808 04:45:52.834709 1679 tablet_service.cc:1467] Processing CreateTablet for tablet 0f129c343fa14118bce173de8cb72bab (DEFAULT_TABLE table=test-table [id=1976cafadc874b98b29a85ccd9e0f2c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:52.834694 1491 tablet_service.cc:1467] Processing CreateTablet for tablet 0f129c343fa14118bce173de8cb72bab (DEFAULT_TABLE table=test-table [id=1976cafadc874b98b29a85ccd9e0f2c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:52.834966 1491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f129c343fa14118bce173de8cb72bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:52.834966 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f129c343fa14118bce173de8cb72bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:52.836943 1875 tablet_service.cc:1467] Processing CreateTablet for tablet 0f129c343fa14118bce173de8cb72bab (DEFAULT_TABLE table=test-table [id=1976cafadc874b98b29a85ccd9e0f2c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:52.837196 1875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f129c343fa14118bce173de8cb72bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:52.837153 1270 catalog_manager.cc:4841] TS c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877): tablet 512b475ecd3f432c89f6f229e7f62410 (table test-table [id=1e0a928c471b47268007978b16bcde59]) successfully deleted
I20260808 04:45:52.837242 2017 tablet_bootstrap.cc:492] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: Bootstrap starting.
I20260808 04:45:52.837448 2018 tablet_bootstrap.cc:492] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: Bootstrap starting.
I20260808 04:45:52.838269 2017 tablet_bootstrap.cc:654] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:52.838356 2018 tablet_bootstrap.cc:654] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:52.839046 1269 catalog_manager.cc:4841] TS c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707): tablet 512b475ecd3f432c89f6f229e7f62410 (table test-table [id=1e0a928c471b47268007978b16bcde59]) successfully deleted
I20260808 04:45:52.839805 2016 tablet_bootstrap.cc:492] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: Bootstrap starting.
I20260808 04:45:52.839895 2018 tablet_bootstrap.cc:492] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: No bootstrap required, opened a new log
I20260808 04:45:52.839977 2018 ts_tablet_manager.cc:1397] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 04:45:52.840444 2018 raft_consensus.cc:348] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:52.840575 2018 raft_consensus.cc:374] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:52.840610 2018 raft_consensus.cc:729] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91961de15e34721bb99afeb90e13dc9, State: Initialized, Role: FOLLOWER
I20260808 04:45:52.840745 2016 tablet_bootstrap.cc:654] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:52.840737 2018 consensus_queue.cc:260] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:52.840999 2018 ts_tablet_manager.cc:1428] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:52.842011 2016 tablet_bootstrap.cc:492] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: No bootstrap required, opened a new log
I20260808 04:45:52.842093 2016 ts_tablet_manager.cc:1397] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:52.842561 2016 raft_consensus.cc:348] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:52.842684 2016 raft_consensus.cc:374] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:52.842725 2016 raft_consensus.cc:729] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba0e2ec56434d59a761b0ec93b95792, State: Initialized, Role: FOLLOWER
I20260808 04:45:52.842872 2016 consensus_queue.cc:260] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:52.843230 2016 ts_tablet_manager.cc:1428] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:52.843914 2049 ts_tablet_manager.cc:1918] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:52.844050 2049 log.cc:1196] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/512b475ecd3f432c89f6f229e7f62410
I20260808 04:45:52.844442 2049 ts_tablet_manager.cc:1939] T 512b475ecd3f432c89f6f229e7f62410 P aba0e2ec56434d59a761b0ec93b95792: Deleting consensus metadata
I20260808 04:45:52.845857 1270 catalog_manager.cc:4841] TS aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025): tablet 512b475ecd3f432c89f6f229e7f62410 (table test-table [id=1e0a928c471b47268007978b16bcde59]) successfully deleted
I20260808 04:45:52.846477 2017 tablet_bootstrap.cc:492] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: No bootstrap required, opened a new log
I20260808 04:45:52.846586 2017 ts_tablet_manager.cc:1397] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20260808 04:45:52.847298 2017 raft_consensus.cc:348] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:52.847409 2017 raft_consensus.cc:374] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:52.847430 2017 raft_consensus.cc:729] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c479eb479bde4c1fb407d952b1bf7987, State: Initialized, Role: FOLLOWER
I20260808 04:45:52.847596 2017 consensus_queue.cc:260] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:52.847882 2017 ts_tablet_manager.cc:1428] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 04:45:53.027359 2025 raft_consensus.cc:482] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:53.027509 2025 raft_consensus.cc:504] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.028786 2025 leader_election.cc:288] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:53.033985 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.029186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:59667 (local address 127.22.2.130:41877)
0808 04:45:53.029332 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.029338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.029578 (+ 240us) server_negotiation.cc:407] Connection header received
0808 04:45:53.029793 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.029796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.029846 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.029895 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:53.030369 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.031074 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.032682 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.033290 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.033413 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.033470 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.033784 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.033860 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:45:53.034418 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.029206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41877 (local address 127.22.2.129:59667)
0808 04:45:53.029477 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:53.029494 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:53.029663 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:53.030063 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:53.030066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:53.030084 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:53.030212 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.030218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.031246 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.031249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.032462 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.032474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.033548 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.033551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.033655 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.033677 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 04:45:53.034173 (+ 496us) client_negotiation.cc:240] Negotiation successful
0808 04:45:53.034246 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":8,"spinlock_wait_cycles":5504,"thread_start_us":90,"threads_started":1}
I20260808 04:45:53.035109 1708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f129c343fa14118bce173de8cb72bab" candidate_uuid: "c479eb479bde4c1fb407d952b1bf7987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c91961de15e34721bb99afeb90e13dc9" is_pre_election: true
I20260808 04:45:53.035383 1708 raft_consensus.cc:2454] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c479eb479bde4c1fb407d952b1bf7987 in term 0.
I20260808 04:45:53.035439 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.029785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:42755 (local address 127.22.2.131:42025)
0808 04:45:53.030154 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.030161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.030179 (+ 18us) server_negotiation.cc:407] Connection header received
0808 04:45:53.030239 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.030242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.030299 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.030383 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:45:53.030646 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.031510 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.033549 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.034307 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.034415 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.034487 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.035207 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.035296 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260808 04:45:53.035833 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.029660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:42025 (local address 127.22.2.129:42755)
0808 04:45:53.029983 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:53.029997 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:53.030111 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:53.030391 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:53.030394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:53.030409 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:45:53.030542 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.030548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.031879 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.031885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.033377 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.033391 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.034889 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.034894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.035040 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.035072 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 04:45:53.035171 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 04:45:53.035364 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"spinlock_wait_cycles":2304,"thread_start_us":92,"threads_started":1}
I20260808 04:45:53.035825 1425 leader_election.cc:302] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c479eb479bde4c1fb407d952b1bf7987, c91961de15e34721bb99afeb90e13dc9; no voters:
I20260808 04:45:53.036342 2025 raft_consensus.cc:2781] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:53.036401 2025 raft_consensus.cc:482] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:53.036437 2025 raft_consensus.cc:3037] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.036680 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f129c343fa14118bce173de8cb72bab" candidate_uuid: "c479eb479bde4c1fb407d952b1bf7987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792" is_pre_election: true
I20260808 04:45:53.036844 1903 raft_consensus.cc:2454] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c479eb479bde4c1fb407d952b1bf7987 in term 0.
I20260808 04:45:53.037671 2025 raft_consensus.cc:504] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.037999 2025 leader_election.cc:288] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [CANDIDATE]: Term 1 election: Requested vote from peers c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:53.038491 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f129c343fa14118bce173de8cb72bab" candidate_uuid: "c479eb479bde4c1fb407d952b1bf7987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792"
I20260808 04:45:53.038628 1903 raft_consensus.cc:3037] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.038686 1708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f129c343fa14118bce173de8cb72bab" candidate_uuid: "c479eb479bde4c1fb407d952b1bf7987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c91961de15e34721bb99afeb90e13dc9"
I20260808 04:45:53.038862 1708 raft_consensus.cc:3037] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.039618 1903 raft_consensus.cc:2454] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c479eb479bde4c1fb407d952b1bf7987 in term 1.
I20260808 04:45:53.039877 1708 raft_consensus.cc:2454] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c479eb479bde4c1fb407d952b1bf7987 in term 1.
I20260808 04:45:53.039902 1425 leader_election.cc:302] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aba0e2ec56434d59a761b0ec93b95792, c479eb479bde4c1fb407d952b1bf7987; no voters:
I20260808 04:45:53.040068 2025 raft_consensus.cc:2781] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:53.040249 2025 raft_consensus.cc:686] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 1 LEADER]: Becoming Leader. State: Replica: c479eb479bde4c1fb407d952b1bf7987, State: Running, Role: LEADER
I20260808 04:45:53.040503 2025 consensus_queue.cc:237] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.042774 1290 catalog_manager.cc:5495] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 reported cstate change: term changed from 0 to 1, leader changed from <none> to c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "c479eb479bde4c1fb407d952b1bf7987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } health_report { overall_health: UNKNOWN } } }
Aug 08 04:45:53 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164351, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:53.110446 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.102244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42255 (local address 127.0.0.1:45920)
0808 04:45:53.102519 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:53.102537 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:53.102657 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:53.102917 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:53.102919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:53.103337 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:53.103448 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.103454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.104198 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.104201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.104833 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.104842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.105362 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.105365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.105429 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.105967 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:53.105979 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:53.107460 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:53.109073 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:53.109082 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:53.109088 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:53.109314 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:53.109566 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:53.109568 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:53.109570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:53.109647 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:53.109989 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:53.109994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:53.110096 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 04:45:53.110189 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 04:45:53.110313 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":112,"threads_started":1}
I20260808 04:45:53.110492 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.102284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.22.2.190:42255)
0808 04:45:53.102530 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.102536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.102622 (+ 86us) server_negotiation.cc:407] Connection header received
0808 04:45:53.102741 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.102743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.102785 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.102871 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:53.103558 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.104091 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.104936 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.105169 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.105249 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.107591 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:53.107594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:53.107596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:53.107617 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:53.108974 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:53.109404 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:53.109406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:53.109407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:53.109486 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:53.109733 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:53.109735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:53.109736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:53.109899 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:53.109973 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.110251 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.110378 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
I20260808 04:45:53.114778 1290 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:45920:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 04:45:53.115523 1290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:53.120862 1491 tablet_service.cc:1467] Processing CreateTablet for tablet 68fb48f318554bc5a1db64a0ad7e1413 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4089a91ff275466c8b429005c2ef6619]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:53.120908 1679 tablet_service.cc:1467] Processing CreateTablet for tablet 68fb48f318554bc5a1db64a0ad7e1413 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4089a91ff275466c8b429005c2ef6619]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:53.120967 1875 tablet_service.cc:1467] Processing CreateTablet for tablet 68fb48f318554bc5a1db64a0ad7e1413 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4089a91ff275466c8b429005c2ef6619]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:45:53.121295 1491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68fb48f318554bc5a1db64a0ad7e1413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.121311 1875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68fb48f318554bc5a1db64a0ad7e1413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.121311 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68fb48f318554bc5a1db64a0ad7e1413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.123759 2017 tablet_bootstrap.cc:492] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: Bootstrap starting.
I20260808 04:45:53.124128 2018 tablet_bootstrap.cc:492] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: Bootstrap starting.
I20260808 04:45:53.124152 2016 tablet_bootstrap.cc:492] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: Bootstrap starting.
I20260808 04:45:53.124761 2017 tablet_bootstrap.cc:654] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.125064 2018 tablet_bootstrap.cc:654] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.125142 2016 tablet_bootstrap.cc:654] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.126392 2016 tablet_bootstrap.cc:492] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: No bootstrap required, opened a new log
I20260808 04:45:53.126405 2018 tablet_bootstrap.cc:492] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: No bootstrap required, opened a new log
I20260808 04:45:53.126405 2017 tablet_bootstrap.cc:492] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: No bootstrap required, opened a new log
I20260808 04:45:53.126495 2018 ts_tablet_manager.cc:1397] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:53.126495 2017 ts_tablet_manager.cc:1397] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:53.126502 2016 ts_tablet_manager.cc:1397] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 04:45:53.126929 2018 raft_consensus.cc:348] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.126929 2017 raft_consensus.cc:348] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.127053 2018 raft_consensus.cc:374] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.127053 2017 raft_consensus.cc:374] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.127002 2016 raft_consensus.cc:348] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.127096 2018 raft_consensus.cc:729] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91961de15e34721bb99afeb90e13dc9, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.127096 2017 raft_consensus.cc:729] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c479eb479bde4c1fb407d952b1bf7987, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.127115 2016 raft_consensus.cc:374] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.127151 2016 raft_consensus.cc:729] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba0e2ec56434d59a761b0ec93b95792, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.127226 2017 consensus_queue.cc:260] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.127226 2018 consensus_queue.cc:260] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.127290 2016 consensus_queue.cc:260] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.127499 2018 ts_tablet_manager.cc:1428] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:53.127499 2017 ts_tablet_manager.cc:1428] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:45:53.127566 2016 ts_tablet_manager.cc:1428] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:45:53.127701 2025 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.127851 2025 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:53.128057 2039 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.128191 2039 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:53.128466 2023 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.128638 2023 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:53.251353 2039 raft_consensus.cc:482] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:53.251487 2039 raft_consensus.cc:504] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.251917 2039 leader_election.cc:288] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:53.252434 1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68fb48f318554bc5a1db64a0ad7e1413" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c479eb479bde4c1fb407d952b1bf7987" is_pre_election: true
I20260808 04:45:53.252478 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68fb48f318554bc5a1db64a0ad7e1413" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792" is_pre_election: true
I20260808 04:45:53.252637 1903 raft_consensus.cc:2454] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 0.
I20260808 04:45:53.252642 1516 raft_consensus.cc:2454] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 0.
I20260808 04:45:53.252959 1620 leader_election.cc:302] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aba0e2ec56434d59a761b0ec93b95792, c91961de15e34721bb99afeb90e13dc9; no voters:
I20260808 04:45:53.253185 2039 raft_consensus.cc:2781] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:53.253268 2039 raft_consensus.cc:482] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:53.253301 2039 raft_consensus.cc:3037] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.254361 2039 raft_consensus.cc:504] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.254679 2039 leader_election.cc:288] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 election: Requested vote from peers c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:53.255012 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68fb48f318554bc5a1db64a0ad7e1413" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792"
I20260808 04:45:53.255014 1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68fb48f318554bc5a1db64a0ad7e1413" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c479eb479bde4c1fb407d952b1bf7987"
I20260808 04:45:53.255138 1903 raft_consensus.cc:3037] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.255138 1516 raft_consensus.cc:3037] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.256242 1903 raft_consensus.cc:2454] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 1.
I20260808 04:45:53.256242 1516 raft_consensus.cc:2454] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 1.
I20260808 04:45:53.256557 1618 leader_election.cc:302] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c479eb479bde4c1fb407d952b1bf7987, c91961de15e34721bb99afeb90e13dc9; no voters:
I20260808 04:45:53.256747 2039 raft_consensus.cc:2781] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:53.256841 2039 raft_consensus.cc:686] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [term 1 LEADER]: Becoming Leader. State: Replica: c91961de15e34721bb99afeb90e13dc9, State: Running, Role: LEADER
I20260808 04:45:53.256976 2039 consensus_queue.cc:237] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.257648 2024 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: TxnStatusTablet state changed. Reason: New leader c91961de15e34721bb99afeb90e13dc9. Latest consensus state: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.257764 2024 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:53.258073 2076 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 04:45:53.258255 2076 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 04:45:53.258754 1287 catalog_manager.cc:5495] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c91961de15e34721bb99afeb90e13dc9 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:53.286275 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.282105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.22.2.130:41877)
0808 04:45:53.282211 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.282216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.282230 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:45:53.282417 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.282420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.282475 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.282552 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:53.282843 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.283610 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.285052 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.285392 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.285495 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.285795 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.285884 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.285958 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.286082 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.286142 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 04:45:53.286396 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.282004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41877 (local address 127.0.0.1:35038)
0808 04:45:53.282149 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:53.282166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:53.282275 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:53.282596 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:53.282599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:53.282616 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:45:53.282726 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.282732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.283845 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.283849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.284907 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.284920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.285507 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.285511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.285602 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.285656 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.285979 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.285993 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 04:45:53.286061 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 04:45:53.286223 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 04:45:53.288872 1516 raft_consensus.cc:1261] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Refusing update from remote peer c91961de15e34721bb99afeb90e13dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:53.288870 1903 raft_consensus.cc:1261] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Refusing update from remote peer c91961de15e34721bb99afeb90e13dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:53.289327 2039 consensus_queue.cc:1035] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:53.289628 2024 consensus_queue.cc:1035] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:53.290884 2023 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: TxnStatusTablet state changed. Reason: New leader c91961de15e34721bb99afeb90e13dc9. Latest consensus state: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.290994 2023 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:53.291186 2025 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: TxnStatusTablet state changed. Reason: New leader c91961de15e34721bb99afeb90e13dc9. Latest consensus state: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.291297 2025 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:53.292793 2027 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.292923 2027 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:53.293232 2025 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.293265 2033 mvcc.cc:205] Tried to move back new op lower bound from 7316129191067226112 to 7316129190942035968. Current Snapshot: MvccSnapshot[applied={T|T < 7316129191067226112}]
I20260808 04:45:53.293329 2025 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P c479eb479bde4c1fb407d952b1bf7987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:53.295667 2039 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.295768 2039 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P c91961de15e34721bb99afeb90e13dc9: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:45:53.296890 2023 tablet_replica.cc:439] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } } }
I20260808 04:45:53.296998 2023 tablet_replica.cc:442] T 68fb48f318554bc5a1db64a0ad7e1413 P aba0e2ec56434d59a761b0ec93b95792: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:45:53.303031 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.298578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60814 (local address 127.22.2.129:43707)
0808 04:45:53.299264 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.299277 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.299292 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:53.299370 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.299373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.299439 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.299566 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:53.299821 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.300620 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.301417 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.301782 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.301887 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.302105 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.302643 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.302782 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.302816 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.302872 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":105,"threads_started":1}
I20260808 04:45:53.314508 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.310444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:46327 (local address 127.22.2.130:41877)
0808 04:45:53.310558 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.310563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.310761 (+ 198us) server_negotiation.cc:407] Connection header received
0808 04:45:53.310916 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.310919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.310972 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.311053 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:53.311311 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.311798 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.313442 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.313665 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.313778 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.314079 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.314146 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.314229 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.314323 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.314371 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 04:45:53.314531 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.310303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41877 (local address 127.22.2.129:46327)
0808 04:45:53.310655 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:53.310668 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:53.310806 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:53.311066 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:53.311069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:53.311077 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:45:53.311203 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.311209 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.311910 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.311914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.313310 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.313321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.313784 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.313787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.313887 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.313929 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.314238 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.314247 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 04:45:53.314313 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 04:45:53.314391 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
I20260808 04:45:53.321282 1903 raft_consensus.cc:1261] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Refusing update from remote peer c479eb479bde4c1fb407d952b1bf7987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:53.321390 1708 raft_consensus.cc:1261] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Refusing update from remote peer c479eb479bde4c1fb407d952b1bf7987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:53.321767 2025 consensus_queue.cc:1035] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:53.322113 2025 consensus_queue.cc:1035] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:53.344779 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.340879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:52367 (local address 127.22.2.129:43707)
0808 04:45:53.340988 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.340992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.341005 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:45:53.341093 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.341097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.341140 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.341235 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:53.341477 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.342242 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.343426 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.343764 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.343870 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.344282 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.344363 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.344434 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.344614 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.344680 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 04:45:53.344807 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.340598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43707 (local address 127.22.2.130:52367)
0808 04:45:53.340887 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:53.340904 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:53.341004 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:53.341247 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:53.341250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:53.341259 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:45:53.341354 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.341360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.342397 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.342400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.343301 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.343311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.343878 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.343881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.343943 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.344025 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.344477 (+ 452us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.344484 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 04:45:53.344574 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:45:53.344656 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260808 04:45:53.370059 1287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:53.370239 1287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:53.372728 1287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: Sending DeleteTablet for 3 replicas of tablet 0f129c343fa14118bce173de8cb72bab
I20260808 04:45:53.373245 1679 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f129c343fa14118bce173de8cb72bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:35024
I20260808 04:45:53.373593 1491 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f129c343fa14118bce173de8cb72bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60812
I20260808 04:45:53.373648 2101 tablet_replica.cc:330] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: stopping tablet replica
I20260808 04:45:53.374030 2101 raft_consensus.cc:2229] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:53.374185 1875 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f129c343fa14118bce173de8cb72bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40186
I20260808 04:45:53.374224 2101 raft_consensus.cc:2258] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:53.375241 2101 ts_tablet_manager.cc:1905] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:53.378822 2101 ts_tablet_manager.cc:1918] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:53.379001 2101 log.cc:1196] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/0f129c343fa14118bce173de8cb72bab
I20260808 04:45:53.379431 2101 ts_tablet_manager.cc:1939] T 0f129c343fa14118bce173de8cb72bab P c91961de15e34721bb99afeb90e13dc9: Deleting consensus metadata
I20260808 04:45:53.379762 2104 tablet_replica.cc:330] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: stopping tablet replica
I20260808 04:45:53.380730 2104 raft_consensus.cc:2229] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:53.380764 1270 catalog_manager.cc:4841] TS c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877): tablet 0f129c343fa14118bce173de8cb72bab (table test-table [id=1976cafadc874b98b29a85ccd9e0f2c8]) successfully deleted
I20260808 04:45:53.381059 2104 raft_consensus.cc:2258] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:53.381839 2104 ts_tablet_manager.cc:1905] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:53.382954 2103 tablet_replica.cc:330] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: stopping tablet replica
I20260808 04:45:53.383087 2103 raft_consensus.cc:2229] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:53.383447 2103 raft_consensus.cc:2258] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:53.385265 2104 ts_tablet_manager.cc:1918] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:53.385406 2104 log.cc:1196] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/0f129c343fa14118bce173de8cb72bab
I20260808 04:45:53.385761 2104 ts_tablet_manager.cc:1939] T 0f129c343fa14118bce173de8cb72bab P aba0e2ec56434d59a761b0ec93b95792: Deleting consensus metadata
I20260808 04:45:53.385999 2103 ts_tablet_manager.cc:1905] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:53.386623 1270 catalog_manager.cc:4841] TS aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025): tablet 0f129c343fa14118bce173de8cb72bab (table test-table [id=1976cafadc874b98b29a85ccd9e0f2c8]) successfully deleted
I20260808 04:45:53.388850 2103 ts_tablet_manager.cc:1918] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:53.388931 2103 log.cc:1196] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/0f129c343fa14118bce173de8cb72bab
I20260808 04:45:53.389226 2103 ts_tablet_manager.cc:1939] T 0f129c343fa14118bce173de8cb72bab P c479eb479bde4c1fb407d952b1bf7987: Deleting consensus metadata
I20260808 04:45:53.389938 1269 catalog_manager.cc:4841] TS c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707): tablet 0f129c343fa14118bce173de8cb72bab (table test-table [id=1976cafadc874b98b29a85ccd9e0f2c8]) successfully deleted
Aug 08 04:45:53 dist-test-slave-04bb krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:45:53 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:53 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 04:45:53.417102 2066 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:53.408039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.22.2.190:42255)
0808 04:45:53.408182 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.408186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.408877 (+ 691us) server_negotiation.cc:407] Connection header received
0808 04:45:53.409016 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.409020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.409081 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.409253 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:53.410112 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.411069 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.412314 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.412769 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.412911 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.416963 (+ 4052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45928: BlockingRecv error: recv got EOF from 127.0.0.1:45928 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 04:45:53 dist-test-slave-04bb krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 04:45:53.430480 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.419465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.22.2.190:42255)
0808 04:45:53.419608 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.419612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.419627 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:53.419662 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.419665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.419709 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.419785 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:53.420801 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.421619 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.422795 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.423200 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.423325 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.426400 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:53.426406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:53.426409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:53.426441 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:53.428430 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:53.428951 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:53.428956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:53.428958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:53.429069 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:53.429522 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:53.429526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:53.429528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:53.429767 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:53.429902 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.430180 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.430310 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 04:45:53.433943 1287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:45:53.434415 1287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:53.439659 1491 tablet_service.cc:1467] Processing CreateTablet for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 (DEFAULT_TABLE table=test-table [id=ea000a11b7d94f90a2aaa3dd18de8a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:53.440011 1491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.441051 1875 tablet_service.cc:1467] Processing CreateTablet for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 (DEFAULT_TABLE table=test-table [id=ea000a11b7d94f90a2aaa3dd18de8a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:53.441346 1875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.442656 2017 tablet_bootstrap.cc:492] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: Bootstrap starting.
I20260808 04:45:53.442876 1679 tablet_service.cc:1467] Processing CreateTablet for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 (DEFAULT_TABLE table=test-table [id=ea000a11b7d94f90a2aaa3dd18de8a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:53.443190 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.443666 2017 tablet_bootstrap.cc:654] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.443828 2016 tablet_bootstrap.cc:492] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: Bootstrap starting.
I20260808 04:45:53.444794 2016 tablet_bootstrap.cc:654] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.445758 2018 tablet_bootstrap.cc:492] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: Bootstrap starting.
I20260808 04:45:53.446552 2018 tablet_bootstrap.cc:654] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.446923 2017 tablet_bootstrap.cc:492] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: No bootstrap required, opened a new log
I20260808 04:45:53.446998 2017 ts_tablet_manager.cc:1397] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 04:45:53.447368 2016 tablet_bootstrap.cc:492] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: No bootstrap required, opened a new log
I20260808 04:45:53.447445 2016 ts_tablet_manager.cc:1397] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 04:45:53.447454 2017 raft_consensus.cc:348] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.447572 2017 raft_consensus.cc:374] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.447588 2018 tablet_bootstrap.cc:492] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: No bootstrap required, opened a new log
I20260808 04:45:53.447604 2017 raft_consensus.cc:729] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c479eb479bde4c1fb407d952b1bf7987, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.447657 2018 ts_tablet_manager.cc:1397] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:53.447724 2017 consensus_queue.cc:260] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.447918 2016 raft_consensus.cc:348] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.448002 2017 ts_tablet_manager.cc:1428] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:53.448019 2016 raft_consensus.cc:374] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.448064 2016 raft_consensus.cc:729] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba0e2ec56434d59a761b0ec93b95792, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.448037 2018 raft_consensus.cc:348] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.448155 2018 raft_consensus.cc:374] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.448187 2018 raft_consensus.cc:729] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91961de15e34721bb99afeb90e13dc9, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.448187 2016 consensus_queue.cc:260] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.448309 2018 consensus_queue.cc:260] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.448446 2016 ts_tablet_manager.cc:1428] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:53.448601 2018 ts_tablet_manager.cc:1428] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:53.541669 2097 raft_consensus.cc:482] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:53.541814 2097 raft_consensus.cc:504] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.542282 2097 leader_election.cc:288] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:53.542807 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c4eff7d68284f48b5ad4ae4c8acdcc8" candidate_uuid: "c479eb479bde4c1fb407d952b1bf7987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792" is_pre_election: true
I20260808 04:45:53.542840 1708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c4eff7d68284f48b5ad4ae4c8acdcc8" candidate_uuid: "c479eb479bde4c1fb407d952b1bf7987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c91961de15e34721bb99afeb90e13dc9" is_pre_election: true
I20260808 04:45:53.542979 1903 raft_consensus.cc:2454] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c479eb479bde4c1fb407d952b1bf7987 in term 0.
I20260808 04:45:53.543000 1708 raft_consensus.cc:2454] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c479eb479bde4c1fb407d952b1bf7987 in term 0.
I20260808 04:45:53.543359 1425 leader_election.cc:302] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c479eb479bde4c1fb407d952b1bf7987, c91961de15e34721bb99afeb90e13dc9; no voters:
I20260808 04:45:53.543541 2097 raft_consensus.cc:2781] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:53.543601 2097 raft_consensus.cc:482] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:53.543641 2097 raft_consensus.cc:3037] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.544888 2097 raft_consensus.cc:504] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.545272 2097 leader_election.cc:288] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [CANDIDATE]: Term 1 election: Requested vote from peers c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:53.545710 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c4eff7d68284f48b5ad4ae4c8acdcc8" candidate_uuid: "c479eb479bde4c1fb407d952b1bf7987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792"
I20260808 04:45:53.545711 1708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c4eff7d68284f48b5ad4ae4c8acdcc8" candidate_uuid: "c479eb479bde4c1fb407d952b1bf7987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c91961de15e34721bb99afeb90e13dc9"
I20260808 04:45:53.545843 1708 raft_consensus.cc:3037] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.545843 1903 raft_consensus.cc:3037] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.547003 1903 raft_consensus.cc:2454] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c479eb479bde4c1fb407d952b1bf7987 in term 1.
I20260808 04:45:53.547003 1708 raft_consensus.cc:2454] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c479eb479bde4c1fb407d952b1bf7987 in term 1.
I20260808 04:45:53.547323 1425 leader_election.cc:302] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aba0e2ec56434d59a761b0ec93b95792, c479eb479bde4c1fb407d952b1bf7987; no voters:
I20260808 04:45:53.547539 2097 raft_consensus.cc:2781] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:53.547664 2097 raft_consensus.cc:686] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 1 LEADER]: Becoming Leader. State: Replica: c479eb479bde4c1fb407d952b1bf7987, State: Running, Role: LEADER
I20260808 04:45:53.548069 2097 consensus_queue.cc:237] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.550201 1291 catalog_manager.cc:5495] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 reported cstate change: term changed from 0 to 1, leader changed from <none> to c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "c479eb479bde4c1fb407d952b1bf7987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:53.575412 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.572207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60828 (local address 127.22.2.129:43707)
0808 04:45:53.572352 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.572357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.572370 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:45:53.572419 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.572422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.572467 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.572588 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:53.572932 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.573528 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.574344 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.574636 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.574761 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.574966 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.575061 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.575125 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.575225 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.575275 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 04:45:53.577623 1903 raft_consensus.cc:1261] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Refusing update from remote peer c479eb479bde4c1fb407d952b1bf7987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:53.577725 1708 raft_consensus.cc:1261] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Refusing update from remote peer c479eb479bde4c1fb407d952b1bf7987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:53.578164 2025 consensus_queue.cc:1035] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:53.578353 2097 consensus_queue.cc:1035] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:53.585799 1291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:53.585914 1291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:53.587672 1291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: Sending DeleteTablet for 3 replicas of tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8
I20260808 04:45:53.588191 1491 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60812
I20260808 04:45:53.588253 1875 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40186
I20260808 04:45:53.588428 2103 tablet_replica.cc:330] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: stopping tablet replica
I20260808 04:45:53.588461 2104 tablet_replica.cc:330] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: stopping tablet replica
I20260808 04:45:53.588459 1679 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:35024
I20260808 04:45:53.588572 2103 raft_consensus.cc:2229] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:53.588635 2101 tablet_replica.cc:330] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: stopping tablet replica
I20260808 04:45:53.588624 2104 raft_consensus.cc:2229] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:53.588734 2101 raft_consensus.cc:2229] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:53.588757 2103 raft_consensus.cc:2258] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:53.588769 2104 raft_consensus.cc:2258] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:53.588873 2101 raft_consensus.cc:2258] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:53.589494 2104 ts_tablet_manager.cc:1905] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:53.589498 2101 ts_tablet_manager.cc:1905] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:53.589498 2103 ts_tablet_manager.cc:1905] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:53.591651 2104 ts_tablet_manager.cc:1918] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:53.591755 2104 log.cc:1196] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/7c4eff7d68284f48b5ad4ae4c8acdcc8
I20260808 04:45:53.591897 2103 ts_tablet_manager.cc:1918] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:53.591897 2101 ts_tablet_manager.cc:1918] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:45:53.591980 2101 log.cc:1196] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/7c4eff7d68284f48b5ad4ae4c8acdcc8
I20260808 04:45:53.591980 2103 log.cc:1196] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/7c4eff7d68284f48b5ad4ae4c8acdcc8
I20260808 04:45:53.592089 2104 ts_tablet_manager.cc:1939] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P aba0e2ec56434d59a761b0ec93b95792: Deleting consensus metadata
I20260808 04:45:53.592260 2101 ts_tablet_manager.cc:1939] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c91961de15e34721bb99afeb90e13dc9: Deleting consensus metadata
I20260808 04:45:53.592278 2103 ts_tablet_manager.cc:1939] T 7c4eff7d68284f48b5ad4ae4c8acdcc8 P c479eb479bde4c1fb407d952b1bf7987: Deleting consensus metadata
I20260808 04:45:53.592862 1270 catalog_manager.cc:4841] TS aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025): tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 (table test-table [id=ea000a11b7d94f90a2aaa3dd18de8a1a]) successfully deleted
I20260808 04:45:53.592968 1291 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 04:45:53.593071 1270 catalog_manager.cc:4841] TS c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877): tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 (table test-table [id=ea000a11b7d94f90a2aaa3dd18de8a1a]) successfully deleted
W20260808 04:45:53.593403 1291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:45:53.594205 1269 catalog_manager.cc:4841] TS c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707): tablet 7c4eff7d68284f48b5ad4ae4c8acdcc8 (table test-table [id=ea000a11b7d94f90a2aaa3dd18de8a1a]) successfully deleted
I20260808 04:45:53.597637 1491 tablet_service.cc:1467] Processing CreateTablet for tablet ff0169ba260444b49dd2b6afe15274f1 (DEFAULT_TABLE table=test-table [id=4f81185255994a0f977cbbe1282ae7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:53.597796 1679 tablet_service.cc:1467] Processing CreateTablet for tablet ff0169ba260444b49dd2b6afe15274f1 (DEFAULT_TABLE table=test-table [id=4f81185255994a0f977cbbe1282ae7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:53.597903 1491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff0169ba260444b49dd2b6afe15274f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.597916 1875 tablet_service.cc:1467] Processing CreateTablet for tablet ff0169ba260444b49dd2b6afe15274f1 (DEFAULT_TABLE table=test-table [id=4f81185255994a0f977cbbe1282ae7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:45:53.598075 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff0169ba260444b49dd2b6afe15274f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.598145 1875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff0169ba260444b49dd2b6afe15274f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:53.599934 2018 tablet_bootstrap.cc:492] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: Bootstrap starting.
I20260808 04:45:53.600317 2016 tablet_bootstrap.cc:492] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: Bootstrap starting.
I20260808 04:45:53.600968 2018 tablet_bootstrap.cc:654] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.601162 2016 tablet_bootstrap.cc:654] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.601580 2017 tablet_bootstrap.cc:492] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987: Bootstrap starting.
I20260808 04:45:53.602348 2017 tablet_bootstrap.cc:654] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:53.602411 2016 tablet_bootstrap.cc:492] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: No bootstrap required, opened a new log
I20260808 04:45:53.602522 2016 ts_tablet_manager.cc:1397] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:53.603027 2016 raft_consensus.cc:348] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.603157 2016 raft_consensus.cc:374] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.603193 2016 raft_consensus.cc:729] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba0e2ec56434d59a761b0ec93b95792, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.603339 2016 consensus_queue.cc:260] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.603513 2018 tablet_bootstrap.cc:492] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: No bootstrap required, opened a new log
I20260808 04:45:53.603606 2018 ts_tablet_manager.cc:1397] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 04:45:53.603626 2016 ts_tablet_manager.cc:1428] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:45:53.604029 2018 raft_consensus.cc:348] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.604154 2018 raft_consensus.cc:374] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.604199 2018 raft_consensus.cc:729] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91961de15e34721bb99afeb90e13dc9, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.604350 2018 consensus_queue.cc:260] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.605098 2017 tablet_bootstrap.cc:492] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987: No bootstrap required, opened a new log
I20260808 04:45:53.605199 2017 ts_tablet_manager.cc:1397] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 04:45:53.605635 2017 raft_consensus.cc:348] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.605774 2017 raft_consensus.cc:374] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:53.605832 2017 raft_consensus.cc:729] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c479eb479bde4c1fb407d952b1bf7987, State: Initialized, Role: FOLLOWER
I20260808 04:45:53.605960 2017 consensus_queue.cc:260] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.606268 2017 ts_tablet_manager.cc:1428] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:45:53.606580 2018 ts_tablet_manager.cc:1428] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 04:45:53.868728 2039 raft_consensus.cc:482] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:45:53.868886 2039 raft_consensus.cc:504] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.869454 2039 leader_election.cc:288] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:53.870007 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0169ba260444b49dd2b6afe15274f1" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792" is_pre_election: true
I20260808 04:45:53.870018 1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0169ba260444b49dd2b6afe15274f1" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c479eb479bde4c1fb407d952b1bf7987" is_pre_election: true
I20260808 04:45:53.870182 1903 raft_consensus.cc:2454] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 0.
I20260808 04:45:53.870216 1516 raft_consensus.cc:2454] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 0.
I20260808 04:45:53.870584 1620 leader_election.cc:302] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aba0e2ec56434d59a761b0ec93b95792, c91961de15e34721bb99afeb90e13dc9; no voters:
I20260808 04:45:53.870831 2039 raft_consensus.cc:2781] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:45:53.870903 2039 raft_consensus.cc:482] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:45:53.870941 2039 raft_consensus.cc:3037] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.871953 2039 raft_consensus.cc:504] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.872277 2039 leader_election.cc:288] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 election: Requested vote from peers c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707), aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025)
I20260808 04:45:53.872670 1903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0169ba260444b49dd2b6afe15274f1" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba0e2ec56434d59a761b0ec93b95792"
I20260808 04:45:53.872668 1516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0169ba260444b49dd2b6afe15274f1" candidate_uuid: "c91961de15e34721bb99afeb90e13dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c479eb479bde4c1fb407d952b1bf7987"
I20260808 04:45:53.872771 1903 raft_consensus.cc:3037] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.872802 1516 raft_consensus.cc:3037] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:53.873683 1903 raft_consensus.cc:2454] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 1.
I20260808 04:45:53.873739 1516 raft_consensus.cc:2454] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91961de15e34721bb99afeb90e13dc9 in term 1.
I20260808 04:45:53.874027 1620 leader_election.cc:302] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aba0e2ec56434d59a761b0ec93b95792, c91961de15e34721bb99afeb90e13dc9; no voters:
I20260808 04:45:53.874223 2039 raft_consensus.cc:2781] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:53.874439 2039 raft_consensus.cc:686] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 1 LEADER]: Becoming Leader. State: Replica: c91961de15e34721bb99afeb90e13dc9, State: Running, Role: LEADER
I20260808 04:45:53.874607 2039 consensus_queue.cc:237] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } }
I20260808 04:45:53.876214 1287 catalog_manager.cc:5495] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c91961de15e34721bb99afeb90e13dc9 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "c91961de15e34721bb99afeb90e13dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c91961de15e34721bb99afeb90e13dc9" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 } health_report { overall_health: UNKNOWN } } }
I20260808 04:45:53.902590 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.899609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35050 (local address 127.22.2.130:41877)
0808 04:45:53.899842 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.899845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.899857 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:45:53.899895 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.899897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.899939 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.900011 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:53.900227 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.900818 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.901614 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.901858 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.901949 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.902207 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.902284 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.902344 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.902428 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.902473 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":105,"threads_started":1}
I20260808 04:45:53.911403 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.908454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:60253 (local address 127.22.2.130:41877)
0808 04:45:53.908627 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:53.908631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:53.908746 (+ 115us) server_negotiation.cc:407] Connection header received
0808 04:45:53.908887 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:53.908890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:53.908932 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:53.908993 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:45:53.909231 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.909699 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.910488 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.910695 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.910774 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.911011 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.911072 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.911151 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:53.911221 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 04:45:53.911273 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":44}
I20260808 04:45:53.911425 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.908352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41877 (local address 127.22.2.130:60253)
0808 04:45:53.908662 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:53.908675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:53.908795 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:53.909019 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:53.909021 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:53.909027 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:45:53.909113 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.909117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.909786 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.909789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.910386 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:53.910394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:53.910810 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:53.910813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:53.910888 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:53.910912 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:45:53.911161 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:45:53.911166 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:45:53.911212 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 04:45:53.911297 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":18,"thread_start_us":110,"threads_started":1}
I20260808 04:45:53.917095 1516 raft_consensus.cc:1261] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Refusing update from remote peer c91961de15e34721bb99afeb90e13dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:53.917181 1903 raft_consensus.cc:1261] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Refusing update from remote peer c91961de15e34721bb99afeb90e13dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:45:53.917505 2142 consensus_queue.cc:1035] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c479eb479bde4c1fb407d952b1bf7987" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:53.917686 2039 consensus_queue.cc:1035] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aba0e2ec56434d59a761b0ec93b95792" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:45:53.951237 1287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:53.951383 1287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:45:53.953349 1287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5edbecaf380b45d59eb35bbaeab6f3b6: Sending DeleteTablet for 3 replicas of tablet ff0169ba260444b49dd2b6afe15274f1
I20260808 04:45:53.953877 1491 tablet_service.cc:1514] Processing DeleteTablet for tablet ff0169ba260444b49dd2b6afe15274f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60812
I20260808 04:45:53.953897 1678 tablet_service.cc:1514] Processing DeleteTablet for tablet ff0169ba260444b49dd2b6afe15274f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:35024
I20260808 04:45:53.954077 1875 tablet_service.cc:1514] Processing DeleteTablet for tablet ff0169ba260444b49dd2b6afe15274f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:45:53 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40186
I20260808 04:45:53.954269 2101 tablet_replica.cc:330] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: stopping tablet replica
I20260808 04:45:53.954434 2101 raft_consensus.cc:2229] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:45:53.954526 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 1397
I20260808 04:45:53.954656 2101 raft_consensus.cc:2258] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:53.954838 2103 tablet_replica.cc:330] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987: stopping tablet replica
I20260808 04:45:53.954968 2103 raft_consensus.cc:2229] T ff0169ba260444b49dd2b6afe15274f1 P c479eb479bde4c1fb407d952b1bf7987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:53.957077 2104 tablet_replica.cc:330] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: stopping tablet replica
I20260808 04:45:53.957181 2104 raft_consensus.cc:2229] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:45:53.957270 2104 raft_consensus.cc:2258] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:45:53.960196 2101 ts_tablet_manager.cc:1905] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:53.963095 2104 ts_tablet_manager.cc:1905] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:45:53.963145 2101 ts_tablet_manager.cc:1918] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:53.963212 2101 log.cc:1196] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/ff0169ba260444b49dd2b6afe15274f1
I20260808 04:45:53.963495 2101 ts_tablet_manager.cc:1939] T ff0169ba260444b49dd2b6afe15274f1 P c91961de15e34721bb99afeb90e13dc9: Deleting consensus metadata
I20260808 04:45:53.964223 1270 catalog_manager.cc:4841] TS c91961de15e34721bb99afeb90e13dc9 (127.22.2.130:41877): tablet ff0169ba260444b49dd2b6afe15274f1 (table test-table [id=4f81185255994a0f977cbbe1282ae7e6]) successfully deleted
I20260808 04:45:53.965517 2104 ts_tablet_manager.cc:1918] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:45:53.965584 2104 log.cc:1196] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/ff0169ba260444b49dd2b6afe15274f1
I20260808 04:45:53.965865 2104 ts_tablet_manager.cc:1939] T ff0169ba260444b49dd2b6afe15274f1 P aba0e2ec56434d59a761b0ec93b95792: Deleting consensus metadata
I20260808 04:45:53.966511 1270 catalog_manager.cc:4841] TS aba0e2ec56434d59a761b0ec93b95792 (127.22.2.131:42025): tablet ff0169ba260444b49dd2b6afe15274f1 (table test-table [id=4f81185255994a0f977cbbe1282ae7e6]) successfully deleted
W20260808 04:45:53.975253 1269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:43707 (error 108)
I20260808 04:45:53.975581 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 1592
I20260808 04:45:53.980146 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:53.978636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43707 (local address 127.0.0.1:60834)
0808 04:45:53.979893 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:53.980009 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1143,"thread_start_us":61,"threads_started":1}
W20260808 04:45:53.980255 1269 catalog_manager.cc:4584] TS c479eb479bde4c1fb407d952b1bf7987 (127.22.2.129:43707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff0169ba260444b49dd2b6afe15274f1: Network error: Client connection negotiation failed: client connection to 127.22.2.129:43707: connect: Connection refused (error 111)
I20260808 04:45:53.988905 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 1786
I20260808 04:45:54.001641 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 1245
2026-08-08T04:45:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3820 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:54 dist-test-slave-04bb krb5kdc[2159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:54 dist-test-slave-04bb krb5kdc[2159](info): set up 2 sockets
Aug 08 04:45:54 dist-test-slave-04bb krb5kdc[2159](info): commencing operation
krb5kdc: starting...
W20260808 04:45:55.241788 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:55 dist-test-slave-04bb krb5kdc[2159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:55.371095 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35153
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:55.483814 2177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:55.484107 2177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:55.484179 2177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:55.488336 2177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:55.488430 2177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:55.488469 2177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:55.488528 2177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:55.488588 2177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:55.493862 2177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:55.494944 2177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:55.495985 2177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:55.504451 2186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:55.504506 2184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:55.504688 2177 server_base.cc:1034] running on GCE node
W20260808 04:45:55.504495 2189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:55.505362 2177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:55.506443 2177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:55.507614 2177 hybrid_clock.cc:648] HybridClock initialized: now 1786164355507582 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T04:45:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1520 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:45:55 dist-test-slave-04bb krb5kdc[2195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:45:55 dist-test-slave-04bb krb5kdc[2195](info): set up 2 sockets
Aug 08 04:45:55 dist-test-slave-04bb krb5kdc[2195](info): commencing operation
krb5kdc: starting...
W20260808 04:45:56.818638 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:45:56 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:45:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:45:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:45:56.943929 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36663
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:36663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:57.059948 2214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:57.060292 2214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:57.060385 2214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:57.064368 2214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:57.064464 2214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:57.064507 2214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:57.064563 2214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:57.064601 2214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:57.069860 2214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:36663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:57.070930 2214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:57.071957 2214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:57.080665 2226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:57.080659 2224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:57.080662 2222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:57.080858 2214 server_base.cc:1034] running on GCE node
I20260808 04:45:57.081619 2214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:57.082792 2214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:57.084007 2214 hybrid_clock.cc:648] HybridClock initialized: now 1786164357083979 us; error 45 us; skew 500 ppm
Aug 08 04:45:57 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164357, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:57.087205 2214 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:57.088224 2214 webserver.cc:460] Webserver started at http://127.22.2.190:37503/ using document root <none> and password file <none>
I20260808 04:45:57.088905 2214 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:57.089008 2214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:57.089274 2214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:57.091619 2214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "5163e0f991d547798b9c0e69c978047b"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "d2997fb8e3a896469f6c7ed2fe9575c0"
server_key_iv: "54e17c148607e472bf9dfb50d6424189"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.092285 2214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "5163e0f991d547798b9c0e69c978047b"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "d2997fb8e3a896469f6c7ed2fe9575c0"
server_key_iv: "54e17c148607e472bf9dfb50d6424189"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.096287 2214 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 04:45:57.099020 2236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.100384 2214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:45:57.100595 2214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "5163e0f991d547798b9c0e69c978047b"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "d2997fb8e3a896469f6c7ed2fe9575c0"
server_key_iv: "54e17c148607e472bf9dfb50d6424189"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.100728 2214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:57.112190 2214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:57.115520 2214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:57.115734 2214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:57.123201 2214 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:36663
I20260808 04:45:57.123232 2314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:36663 every 8 connection(s)
I20260808 04:45:57.124176 2214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:57.127103 2316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:45:57.129890 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 2214
I20260808 04:45:57.130034 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:45:57.130375 22538 external_mini_cluster.cc:1428] Setting key f8b35592c982bc6cb54654f8d4bf5fea
I20260808 04:45:57.133675 2316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Bootstrap starting.
I20260808 04:45:57.135975 2316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Neither blocks nor log segments found. Creating new log.
I20260808 04:45:57.136829 2316 log.cc:824] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:57.138916 2316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: No bootstrap required, opened a new log
Aug 08 04:45:57 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:57.142253 2316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } }
I20260808 04:45:57.142501 2316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:45:57.142599 2316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5163e0f991d547798b9c0e69c978047b, State: Initialized, Role: FOLLOWER
I20260808 04:45:57.143049 2316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } }
I20260808 04:45:57.143203 2316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:57.143294 2316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:57.143420 2316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:45:57.144275 2316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } }
I20260808 04:45:57.144678 2316 leader_election.cc:302] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5163e0f991d547798b9c0e69c978047b; no voters:
I20260808 04:45:57.145049 2316 leader_election.cc:288] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:45:57.145138 2321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:45:57.145335 2321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 1 LEADER]: Becoming Leader. State: Replica: 5163e0f991d547798b9c0e69c978047b, State: Running, Role: LEADER
I20260808 04:45:57.145699 2321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } }
I20260808 04:45:57.146152 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:57.132007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.22.2.190:36663)
0808 04:45:57.132493 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:57.132508 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:57.132625 (+ 117us) server_negotiation.cc:407] Connection header received
0808 04:45:57.133252 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:57.133270 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:57.133619 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:57.134017 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:57.134874 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.135876 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.136911 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.137601 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.137814 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:57.140426 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:57.140432 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:57.140445 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:57.140472 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:57.142916 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:57.143401 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:57.143407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:57.143413 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:57.143576 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:57.143872 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:57.143875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:57.143877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:57.144277 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:57.144498 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:57.144815 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 04:45:57.145133 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":138,"threads_started":1}
I20260808 04:45:57.146339 2316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:57.149296 2324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5163e0f991d547798b9c0e69c978047b. Latest consensus state: current_term: 1 leader_uuid: "5163e0f991d547798b9c0e69c978047b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } } }
I20260808 04:45:57.149315 2322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5163e0f991d547798b9c0e69c978047b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } } }
I20260808 04:45:57.149631 2324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:57.149655 2322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: This master's current role is: LEADER
W20260808 04:45:57.151592 2341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:45:57.151664 2341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:45:57.151749 2342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:57.152415 2342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:57.157874 2342 catalog_manager.cc:1347] Generated new cluster ID: 0d2661e7a7da41178593f4da9c110da1
I20260808 04:45:57.157979 2342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:57.169319 2342 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:45:57.170264 2342 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:57.175623 2342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Generated new TSK 0
I20260808 04:45:57.176331 2342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:45:57.216245 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36663
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:57.329783 2350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:57.330051 2350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:57.330127 2350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:57.334061 2350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:57.334142 2350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:57.334240 2350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:57.339272 2350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:57.340289 2350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:57.341316 2350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:57.351166 2358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:57.351198 2359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:57.351323 2350 server_base.cc:1034] running on GCE node
W20260808 04:45:57.351487 2362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:57.351881 2350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:57.352598 2350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:57.353848 2350 hybrid_clock.cc:648] HybridClock initialized: now 1786164357353827 us; error 33 us; skew 500 ppm
Aug 08 04:45:57 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164357, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:57.357161 2350 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:45:57.358095 2350 webserver.cc:460] Webserver started at http://127.22.2.129:38031/ using document root <none> and password file <none>
I20260808 04:45:57.358682 2350 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:57.358753 2350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:57.358945 2350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:57.361230 2350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "a792a0f935c1485ea4a3e5d669783bba"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "9a9188714d456757358d0ac657575ebd"
server_key_iv: "23a8abcf9acbe0b08f1b95637b54aba7"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.361804 2350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "a792a0f935c1485ea4a3e5d669783bba"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "9a9188714d456757358d0ac657575ebd"
server_key_iv: "23a8abcf9acbe0b08f1b95637b54aba7"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.365850 2350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:45:57.368443 2374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.369891 2350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:45:57.370051 2350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "a792a0f935c1485ea4a3e5d669783bba"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "9a9188714d456757358d0ac657575ebd"
server_key_iv: "23a8abcf9acbe0b08f1b95637b54aba7"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.370183 2350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:57.385617 2350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:57.388657 2350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:57.388891 2350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:57.389542 2350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:57.390470 2350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:57.390527 2350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.390612 2350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:57.390748 2350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 04:45:57.401125 2350 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:39805
I20260808 04:45:57.401129 2520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:39805 every 8 connection(s)
I20260808 04:45:57.402113 2350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:57.403192 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 2350
I20260808 04:45:57.403297 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:45:57.403565 22538 external_mini_cluster.cc:1428] Setting key b0bba25b676f4d7d1fa720ec7d7d7497
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Aug 08 04:45:57 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164357, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 04:45:57.419541 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:57.405233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:33649 (local address 127.22.2.190:36663)
0808 04:45:57.405374 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:57.405380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:57.406443 (+ 1063us) server_negotiation.cc:407] Connection header received
0808 04:45:57.408628 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:57.408631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:57.408679 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:57.408754 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:57.410099 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.411027 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.412219 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.412652 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.412772 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:57.415352 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:57.415355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:57.415358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:57.415383 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:57.416957 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:57.417647 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:57.417654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:57.417656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:57.417764 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:57.418252 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:57.418257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:57.418259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:57.418500 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:57.418666 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:57.419301 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 04:45:57.419416 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 04:45:57.421052 2525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:57.405700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:33649)
0808 04:45:57.406260 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:57.406318 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:57.408315 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:57.408975 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:57.408984 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:57.409407 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:57.409925 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:57.409941 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.411167 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.411171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:57.412079 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:57.412090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.412830 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.412833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:57.412984 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:57.413644 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:57.413660 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:57.415189 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:57.417100 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:57.417105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:57.417125 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:57.417511 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:57.417896 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:57.417900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:57.417902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:57.418071 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:57.418665 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:57.418673 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:57.418938 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 04:45:57.419263 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 04:45:57.419805 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":164,"threads_started":1}
I20260808 04:45:57.422837 2521 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36663
I20260808 04:45:57.423117 2521 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:57.423630 2521 heartbeater.cc:511] Master 127.22.2.190:36663 requested a full tablet report, sending...
I20260808 04:45:57.425474 2269 ts_manager.cc:194] Registered new tserver with Master: a792a0f935c1485ea4a3e5d669783bba (127.22.2.129:39805)
I20260808 04:45:57.427033 2269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:33649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:45:57.435319 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36663
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:57.548496 2531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:57.548823 2531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:57.548868 2531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:57.552882 2531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:57.552933 2531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:57.552989 2531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:57.558068 2531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:57.559051 2531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:57.559993 2531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:57.569715 2541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:57.569820 2531 server_base.cc:1034] running on GCE node
W20260808 04:45:57.569737 2542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:57.569985 2547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:57.570364 2531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:57.570951 2531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:57.572149 2531 hybrid_clock.cc:648] HybridClock initialized: now 1786164357572135 us; error 37 us; skew 500 ppm
Aug 08 04:45:57 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164357, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:57.575037 2531 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:45:57.575906 2531 webserver.cc:460] Webserver started at http://127.22.2.130:42785/ using document root <none> and password file <none>
I20260808 04:45:57.576458 2531 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:57.576561 2531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:57.576769 2531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:57.578764 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "f6bfb77f4c714dc898677837b396088c"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "ff9eaa22438fd36f832f54790a195afe"
server_key_iv: "6b3aa35de21f49c4739bea02db7b9646"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.579272 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "f6bfb77f4c714dc898677837b396088c"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "ff9eaa22438fd36f832f54790a195afe"
server_key_iv: "6b3aa35de21f49c4739bea02db7b9646"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.582890 2531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:45:57.585243 2558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.586459 2531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:57.586552 2531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "f6bfb77f4c714dc898677837b396088c"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "ff9eaa22438fd36f832f54790a195afe"
server_key_iv: "6b3aa35de21f49c4739bea02db7b9646"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.586680 2531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:57.623925 2531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:57.626852 2531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:57.627077 2531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:57.627990 2531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:57.629007 2531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:57.629079 2531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.629145 2531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:57.629201 2531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.637758 2531 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:43757
I20260808 04:45:57.637777 2720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:43757 every 8 connection(s)
I20260808 04:45:57.638682 2531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:57.642040 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 2531
I20260808 04:45:57.642227 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:45:57.642551 22538 external_mini_cluster.cc:1428] Setting key d5b4800869a5f945a9057e53203370d4
Aug 08 04:45:57 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164357, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:57.654295 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:57.640497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:54311 (local address 127.22.2.190:36663)
0808 04:45:57.640674 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:57.640678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:57.641489 (+ 811us) server_negotiation.cc:407] Connection header received
0808 04:45:57.643003 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:57.643007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:57.643054 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:57.643159 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:57.644648 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.645170 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.646516 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.646920 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.647067 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:57.650246 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:57.650253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:57.650257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:57.650291 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:57.651831 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:57.652484 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:57.652487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:57.652488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:57.652621 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:57.653053 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:57.653056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:57.653057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:57.653274 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:57.653359 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:57.654062 (+ 703us) server_negotiation.cc:299] Negotiation successful
0808 04:45:57.654166 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:45:57.655493 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:57.640901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:54311)
0808 04:45:57.641344 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:57.641386 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:57.642785 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:57.643341 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:57.643350 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:57.643820 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:57.644459 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:57.644474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.645311 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.645317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:57.646340 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:57.646355 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.647091 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.647099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:57.647336 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:57.647992 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:57.648019 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:57.650022 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:57.651961 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:57.651970 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:57.651991 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:57.652365 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:57.652721 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:57.652724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:57.652726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:57.652923 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:57.653398 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:57.653407 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:57.653759 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 04:45:57.653998 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 04:45:57.654393 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":102,"threads_started":1}
I20260808 04:45:57.656810 2721 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36663
I20260808 04:45:57.657183 2721 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:45:57.657811 2721 heartbeater.cc:511] Master 127.22.2.190:36663 requested a full tablet report, sending...
I20260808 04:45:57.658967 2269 ts_manager.cc:194] Registered new tserver with Master: f6bfb77f4c714dc898677837b396088c (127.22.2.130:43757)
I20260808 04:45:57.659813 2269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:54311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:45:57.676229 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36663
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:57.789094 2729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:57.789381 2729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:57.789436 2729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:57.793601 2729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:57.793658 2729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:57.793715 2729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:57.798750 2729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:57.799743 2729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:57.800930 2729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:57.811776 2743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:57.811985 2729 server_base.cc:1034] running on GCE node
W20260808 04:45:57.811789 2741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:57.811789 2740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:57.812513 2729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:57.813210 2729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:57.814380 2729 hybrid_clock.cc:648] HybridClock initialized: now 1786164357814362 us; error 30 us; skew 500 ppm
Aug 08 04:45:57 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164357, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:57.817108 2729 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:45:57.818009 2729 webserver.cc:460] Webserver started at http://127.22.2.131:43679/ using document root <none> and password file <none>
I20260808 04:45:57.818578 2729 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:57.818641 2729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:57.818888 2729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:45:57.821007 2729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "6ecc0f8559a0471c9c6ea619e89d65d1"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "2a0530491b4b0556197c9cab16498eaf"
server_key_iv: "155e63f79bbbf21ee295e49b35a0ecdc"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.821583 2729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "6ecc0f8559a0471c9c6ea619e89d65d1"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "2a0530491b4b0556197c9cab16498eaf"
server_key_iv: "155e63f79bbbf21ee295e49b35a0ecdc"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.825701 2729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 04:45:57.828080 2755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.829331 2729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:45:57.829468 2729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "6ecc0f8559a0471c9c6ea619e89d65d1"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "2a0530491b4b0556197c9cab16498eaf"
server_key_iv: "155e63f79bbbf21ee295e49b35a0ecdc"
server_key_version: "encryptionkey@0"
I20260808 04:45:57.829581 2729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:57.846922 2729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:57.850055 2729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:57.850206 2729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:57.850726 2729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:57.851605 2729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:57.851661 2729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.851701 2729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:57.851717 2729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:57.861341 2729 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:34801
I20260808 04:45:57.861362 2924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:34801 every 8 connection(s)
I20260808 04:45:57.862277 2729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:57.863027 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 2729
I20260808 04:45:57.863180 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:45:57.863547 22538 external_mini_cluster.cc:1428] Setting key 002f1a6331612f7c3356b6813c63a485
Aug 08 04:45:57 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164357, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:45:57.877203 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:57.864814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:49065 (local address 127.22.2.190:36663)
0808 04:45:57.864966 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:57.864971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:57.865952 (+ 981us) server_negotiation.cc:407] Connection header received
0808 04:45:57.867465 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:57.867468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:57.867516 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:57.867589 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:57.868872 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.869399 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.870615 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.870835 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.870911 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:57.873524 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:57.873527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:57.873529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:57.873552 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:57.874954 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:57.875508 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:57.875511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:57.875512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:57.875589 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:57.875960 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:57.875963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:57.875964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:57.876133 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:57.876210 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:57.876890 (+ 680us) server_negotiation.cc:299] Negotiation successful
0808 04:45:57.877008 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 04:45:57.878738 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:57.865229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:49065)
0808 04:45:57.865783 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:57.865828 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:57.867258 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:57.867779 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:57.867786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:57.868172 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:57.868725 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:57.868741 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.869519 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.869523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:57.870498 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:57.870509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:57.870965 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:57.870968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:57.871120 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:57.871698 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:57.871727 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:57.873352 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:45:57.875067 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:57.875073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:57.875089 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:57.875398 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:57.875714 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:45:57.875718 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:45:57.875719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:45:57.875869 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:45:57.876259 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:45:57.876274 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:45:57.876576 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 04:45:57.876828 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 04:45:57.877213 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":149,"threads_started":1}
I20260808 04:45:57.879904 2925 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36663
I20260808 04:45:57.880172 2925 heartbeater.cc:464] Registering TS with master...
I20260808 04:45:57.880750 2925 heartbeater.cc:511] Master 127.22.2.190:36663 requested a full tablet report, sending...
I20260808 04:45:57.881852 2269 ts_manager.cc:194] Registered new tserver with Master: 6ecc0f8559a0471c9c6ea619e89d65d1 (127.22.2.131:34801)
I20260808 04:45:57.882499 2269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:49065
I20260808 04:45:57.890283 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
WARNING: no policy specified for oryx/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
WARNING: no policy specified for oryx/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
I20260808 04:45:57.967105 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 2350
I20260808 04:45:57.998136 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 2531
I20260808 04:45:58.013106 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 2729
I20260808 04:45:58.024003 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 2214
I20260808 04:45:58.032847 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36663
--webserver_interface=127.22.2.190
--webserver_port=37503
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:36663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:45:58.149830 2937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:58.150075 2937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:58.150115 2937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:58.154048 2937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:45:58.154100 2937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:58.154120 2937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:58.154136 2937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:45:58.154151 2937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:45:58.159343 2937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:36663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=37503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:58.160406 2937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:58.161458 2937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:58.170234 2947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:58.170312 2950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:58.170418 2937 server_base.cc:1034] running on GCE node
W20260808 04:45:58.170250 2946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:58.171120 2937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:58.172263 2937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:58.173472 2937 hybrid_clock.cc:648] HybridClock initialized: now 1786164358173436 us; error 53 us; skew 500 ppm
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164358, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:58.176348 2937 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:45:58.177260 2937 webserver.cc:460] Webserver started at http://127.22.2.190:37503/ using document root <none> and password file <none>
I20260808 04:45:58.177784 2937 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:58.177855 2937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:58.181684 2937 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 04:45:58.183773 2960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.185045 2937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 04:45:58.185146 2937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "5163e0f991d547798b9c0e69c978047b"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "d2997fb8e3a896469f6c7ed2fe9575c0"
server_key_iv: "54e17c148607e472bf9dfb50d6424189"
server_key_version: "encryptionkey@0"
I20260808 04:45:58.185645 2937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:58.197562 2937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:58.200302 2937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:58.200456 2937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:58.207644 2937 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:36663
I20260808 04:45:58.207731 3027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:36663 every 8 connection(s)
I20260808 04:45:58.208822 2937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:45:58.209594 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 2937
I20260808 04:45:58.210068 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:39805
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:36663
--webserver_port=38031
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.2.129
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:45:58.213897 3028 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:45:58.215631 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Bootstrap starting.
I20260808 04:45:58.232811 3028 log.cc:824] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:45:58.239511 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:45:58.239916 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Bootstrap complete.
I20260808 04:45:58.244316 3028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } }
I20260808 04:45:58.244807 3028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5163e0f991d547798b9c0e69c978047b, State: Initialized, Role: FOLLOWER
I20260808 04:45:58.245316 3028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } }
I20260808 04:45:58.245462 3028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:45:58.245533 3028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:45:58.245661 3028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:45:58.247560 3028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } }
I20260808 04:45:58.247910 3028 leader_election.cc:302] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5163e0f991d547798b9c0e69c978047b; no voters:
I20260808 04:45:58.248308 3028 leader_election.cc:288] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 04:45:58.248395 3033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 04:45:58.248636 3033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [term 2 LEADER]: Becoming Leader. State: Replica: 5163e0f991d547798b9c0e69c978047b, State: Running, Role: LEADER
I20260808 04:45:58.248927 3033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } }
I20260808 04:45:58.249603 3028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:45:58.249845 3033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5163e0f991d547798b9c0e69c978047b. Latest consensus state: current_term: 2 leader_uuid: "5163e0f991d547798b9c0e69c978047b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } } }
I20260808 04:45:58.249963 3033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:58.250412 3040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:45:58.250444 3034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5163e0f991d547798b9c0e69c978047b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5163e0f991d547798b9c0e69c978047b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36663 } } }
I20260808 04:45:58.250664 3034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b [sys.catalog]: This master's current role is: LEADER
I20260808 04:45:58.253768 3040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:45:58.254391 3040 catalog_manager.cc:1259] Loaded cluster ID: 0d2661e7a7da41178593f4da9c110da1
I20260808 04:45:58.254455 3040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:45:58.256851 3040 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:45:58.257494 3040 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5163e0f991d547798b9c0e69c978047b: Loaded TSK: 0
I20260808 04:45:58.258683 3040 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 04:45:58.341024 3031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:58.341291 3031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:58.341336 3031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:58.345399 3031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:58.345458 3031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:58.345543 3031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:45:58.351148 3031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:39805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=38031
--tserver_master_addrs=127.22.2.190:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:58.352242 3031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:58.353473 3031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:58.363461 3066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:58.363463 3067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:58.363618 3031 server_base.cc:1034] running on GCE node
W20260808 04:45:58.363768 3070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:58.364180 3031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:58.364926 3031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:58.366119 3031 hybrid_clock.cc:648] HybridClock initialized: now 1786164358366095 us; error 36 us; skew 500 ppm
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164358, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:58.369043 3031 init.cc:377] Logged in from keytab as oryx/127.22.2.129@KRBTEST.COM (short username oryx)
I20260808 04:45:58.370014 3031 webserver.cc:460] Webserver started at http://127.22.2.129:38031/ using document root <none> and password file <none>
I20260808 04:45:58.370582 3031 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:58.370656 3031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:58.375765 3031 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 04:45:58.378015 3078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.379297 3031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:45:58.379427 3031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "a792a0f935c1485ea4a3e5d669783bba"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "9a9188714d456757358d0ac657575ebd"
server_key_iv: "23a8abcf9acbe0b08f1b95637b54aba7"
server_key_version: "encryptionkey@0"
I20260808 04:45:58.379936 3031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:58.405903 3031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:58.408826 3031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:58.409042 3031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:58.409662 3031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:58.410488 3031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:58.410535 3031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.410614 3031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:58.410645 3031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.419931 3031 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:39805
I20260808 04:45:58.419974 3238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:39805 every 8 connection(s)
I20260808 04:45:58.420974 3031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:45:58.428691 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 3031
I20260808 04:45:58.429208 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:43757
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:36663
--webserver_port=42785
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.2.130
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 04:45:58.434460 3241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.422956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:57683 (local address 127.22.2.190:36663)
0808 04:45:58.423426 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.423446 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.423692 (+ 246us) server_negotiation.cc:407] Connection header received
0808 04:45:58.425172 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.425185 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.425466 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.425733 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.426778 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.427735 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.429003 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.429780 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.429962 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.433792 (+ 3830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:57683: BlockingRecv error: recv got EOF from 127.22.2.129:57683 (error 108)
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":99,"threads_started":1}
I20260808 04:45:58.434691 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.423010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:57683)
0808 04:45:58.423525 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.423567 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.424792 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.425817 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.425824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.426165 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.426618 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.426631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.427865 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.427868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.428874 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.428885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.429958 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.429961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.430096 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.430706 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.430723 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.433580 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":163,"threads_started":1}
W20260808 04:45:58.435307 3239 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:36663 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.2.190:36663: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:58.441617 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.435749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:44057)
0808 04:45:58.435865 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.435881 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.435955 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.436406 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.436408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.436594 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.436691 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.436696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.437808 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.437811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.438351 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.438359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.439016 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.439019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.439083 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.439587 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.439598 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.441451 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 04:45:58.441715 3241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.435954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:44057 (local address 127.22.2.190:36663)
0808 04:45:58.436100 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.436107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.436122 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:58.436208 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.436214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.436278 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.436384 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.436809 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.437670 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.438469 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.438889 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.439013 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.441585 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:44057: BlockingRecv error: recv got EOF from 127.22.2.129:44057 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:58.447604 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.442406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:60721)
0808 04:45:58.442620 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.442637 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.442697 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.442961 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.442963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.443137 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.443209 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.443214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.444195 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.444198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.444762 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.444771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.445337 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.445340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.445402 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.445874 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.445886 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.447525 (+ 1639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145}
W20260808 04:45:58.447780 3241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.442556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:60721 (local address 127.22.2.190:36663)
0808 04:45:58.442692 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.442698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.442716 (+ 18us) server_negotiation.cc:407] Connection header received
0808 04:45:58.442760 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.442763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.442817 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.442914 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.443326 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.444078 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.444862 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.445203 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.445309 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.447674 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:60721: BlockingRecv error: recv got EOF from 127.22.2.129:60721 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 04:45:58.447805 3239 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:45:58.547902 3243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:58.548151 3243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:58.548192 3243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:58.552421 3243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:58.552479 3243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:58.552549 3243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:45:58.557698 3243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:43757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=42785
--tserver_master_addrs=127.22.2.190:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:58.558669 3243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:58.559778 3243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:58.569615 3249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:58.569603 3250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:58.569600 3253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:58.570012 3243 server_base.cc:1034] running on GCE node
I20260808 04:45:58.570431 3243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:58.571040 3243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:58.572201 3243 hybrid_clock.cc:648] HybridClock initialized: now 1786164358572184 us; error 27 us; skew 500 ppm
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164358, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:58.574965 3243 init.cc:377] Logged in from keytab as oryx/127.22.2.130@KRBTEST.COM (short username oryx)
I20260808 04:45:58.575866 3243 webserver.cc:460] Webserver started at http://127.22.2.130:42785/ using document root <none> and password file <none>
I20260808 04:45:58.576434 3243 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:58.576570 3243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:58.580302 3243 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 04:45:58.582422 3266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.583896 3243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 04:45:58.584043 3243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "f6bfb77f4c714dc898677837b396088c"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "ff9eaa22438fd36f832f54790a195afe"
server_key_iv: "6b3aa35de21f49c4739bea02db7b9646"
server_key_version: "encryptionkey@0"
I20260808 04:45:58.584688 3243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:58.601758 3243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:58.604965 3243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:58.605171 3243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:58.605756 3243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:58.606594 3243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:58.606667 3243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.606746 3243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:58.606801 3243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.616267 3243 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:43757
I20260808 04:45:58.616291 3421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:43757 every 8 connection(s)
I20260808 04:45:58.617268 3243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:45:58.617677 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 3243
I20260808 04:45:58.618251 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:34801
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:36663
--webserver_port=43679
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.2.131
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 04:45:58.628808 3241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.619253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:38039 (local address 127.22.2.190:36663)
0808 04:45:58.619435 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.619442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.620274 (+ 832us) server_negotiation.cc:407] Connection header received
0808 04:45:58.621700 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.621707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.621777 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.621884 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.623047 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.623585 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.624802 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.625028 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.625143 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.628681 (+ 3538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:38039: BlockingRecv error: recv got EOF from 127.22.2.130:38039 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 04:45:58.629410 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.619614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:38039)
0808 04:45:58.620105 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.620156 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.621475 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.622042 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.622049 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.622419 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.622891 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.622904 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.623715 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.623718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.624663 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.624673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.625185 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.625189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.625370 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.625996 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.626012 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.628533 (+ 2521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":151,"threads_started":1}
W20260808 04:45:58.629980 3422 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:36663 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.2.190:36663: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:58.635620 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.630469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:51705)
0808 04:45:58.630585 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.630601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.630675 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.630895 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.630897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.631019 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.631103 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.631107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.632096 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.632099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.632689 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.632698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.633283 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.633286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.633367 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.633839 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.633850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.635536 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 04:45:58.636061 3241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.630560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:51705 (local address 127.22.2.190:36663)
0808 04:45:58.630687 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.630691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.630706 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:45:58.630748 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.630751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.630789 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.630872 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.631214 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.631988 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.632793 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.633161 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.633263 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.635946 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:51705: BlockingRecv error: recv got EOF from 127.22.2.130:51705 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:58.641225 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.636106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:48271)
0808 04:45:58.636255 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.636271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.636377 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.636709 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.636712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.636828 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.636896 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.636900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.637919 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.637921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.638411 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.638418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.638946 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.638948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.639001 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.639468 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.639479 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.641145 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260808 04:45:58.641427 3241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.636173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:48271 (local address 127.22.2.190:36663)
0808 04:45:58.636268 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.636272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.636341 (+ 69us) server_negotiation.cc:407] Connection header received
0808 04:45:58.636540 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.636543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.636589 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.636673 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.637012 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.637808 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.638510 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.638843 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.638940 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.641304 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:48271: BlockingRecv error: recv got EOF from 127.22.2.130:48271 (error 108)
Metrics: {"server-negotiator.queue_time_us":37,"thread_start_us":136,"threads_started":1}
W20260808 04:45:58.641518 3422 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:45:58.736814 3427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:45:58.737061 3427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:45:58.737133 3427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:45:58.741053 3427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:45:58.741103 3427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:45:58.741236 3427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:45:58.746381 3427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:34801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=43679
--tserver_master_addrs=127.22.2.190:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:45:58.747419 3427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:45:58.748454 3427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:45:58.758854 3433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:58.758822 3436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:45:58.758837 3439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:45:58.759251 3427 server_base.cc:1034] running on GCE node
I20260808 04:45:58.759671 3427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:45:58.760371 3427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:45:58.761570 3427 hybrid_clock.cc:648] HybridClock initialized: now 1786164358761537 us; error 48 us; skew 500 ppm
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164358, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:45:58.764693 3427 init.cc:377] Logged in from keytab as oryx/127.22.2.131@KRBTEST.COM (short username oryx)
I20260808 04:45:58.765834 3427 webserver.cc:460] Webserver started at http://127.22.2.131:43679/ using document root <none> and password file <none>
I20260808 04:45:58.766458 3427 fs_manager.cc:359] Metadata directory not provided
I20260808 04:45:58.766538 3427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:45:58.770839 3427 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:45:58.773015 3448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.774323 3427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:45:58.774428 3427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "6ecc0f8559a0471c9c6ea619e89d65d1"
format_stamp: "Formatted at 2026-08-08 04:45:57 on dist-test-slave-04bb"
server_key: "2a0530491b4b0556197c9cab16498eaf"
server_key_iv: "155e63f79bbbf21ee295e49b35a0ecdc"
server_key_version: "encryptionkey@0"
I20260808 04:45:58.774961 3427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:45:58.808389 3427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:45:58.811789 3427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:45:58.812023 3427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:45:58.812783 3427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:45:58.813733 3427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:45:58.813787 3427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.813865 3427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:45:58.813905 3427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:45:58.823750 3427 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:34801
I20260808 04:45:58.823771 3600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:34801 every 8 connection(s)
I20260808 04:45:58.824970 3427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:45:58.826156 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 3427
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:58.836669 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.826992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42462 (local address 127.22.2.190:36663)
0808 04:45:58.827145 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.827150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.827498 (+ 348us) server_negotiation.cc:407] Connection header received
0808 04:45:58.827681 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.827687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.827737 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.827845 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.828716 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.829240 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.830310 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.830536 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.830648 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.832264 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:45:58.832270 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:45:58.832282 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:45:58.832310 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:45:58.834208 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:58.834719 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:58.834724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:58.834728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:58.834862 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:45:58.835184 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:45:58.835188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:45:58.835190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:45:58.835654 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:45:58.835898 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 04:45:58.836223 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 04:45:58.836472 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 04:45:58.837522 3428 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.827439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:52419 (local address 127.22.2.190:36663)
0808 04:45:58.827567 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.827572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.828336 (+ 764us) server_negotiation.cc:407] Connection header received
0808 04:45:58.829726 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.829729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.829777 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.829880 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.831067 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.831667 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.833054 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.833393 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.833506 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.837439 (+ 3933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:52419: BlockingRecv error: recv got EOF from 127.22.2.131:52419 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:45:58.838271 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.827606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:52419)
0808 04:45:58.828125 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.828219 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.829518 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.830048 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.830055 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.830436 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.830927 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.830940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.831804 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.831808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.832930 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.832940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.833556 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.833559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.833706 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.834324 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.834351 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.837320 (+ 2969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":156,"threads_started":1}
W20260808 04:45:58.838861 3602 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:36663 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.2.190:36663: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:58.844095 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.839341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:38341)
0808 04:45:58.839530 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.839545 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.839620 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.839862 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.839865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.840014 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.840099 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.840104 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.840885 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.840890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.841430 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.841438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.841890 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.841894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.841986 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.842365 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.842376 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.843961 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91}
W20260808 04:45:58.844184 3241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.839473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:38341 (local address 127.22.2.190:36663)
0808 04:45:58.839604 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.839608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.839620 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:45:58.839712 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.839714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.839751 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.839846 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.840213 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.840779 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.841551 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.841777 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.841886 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.844072 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:38341: BlockingRecv error: recv got EOF from 127.22.2.131:38341 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:58 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:58.849385 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:58.844684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:38375)
0808 04:45:58.844811 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:58.844823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:58.844890 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:58.845136 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:58.845139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:58.845268 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:58.845336 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.845340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.845973 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.845976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.846568 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:58.846577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.846962 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.846965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:58.847025 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.847542 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:58.847555 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:58.849235 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 04:45:58.849447 3241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:58.844784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:38375 (local address 127.22.2.190:36663)
0808 04:45:58.844947 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:58.844951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:58.844971 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:45:58.845000 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:58.845002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:58.845032 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:58.845094 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:58.845434 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.845872 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.846673 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:58.846865 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:58.846935 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:58.849346 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:38375: BlockingRecv error: recv got EOF from 127.22.2.131:38375 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260808 04:45:58.849646 3602 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 04:45:59 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:59 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:59.455447 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:59.448515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:48933)
0808 04:45:59.448862 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:59.448884 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:59.448997 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:59.449330 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:59.449332 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:59.449522 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:59.449626 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:59.449631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.450854 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.450858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:59.451766 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:59.451778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.452444 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.452447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:59.452514 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:59.453206 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:59.453225 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:59.455319 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
W20260808 04:45:59.455693 3611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:59.448607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48933 (local address 127.22.2.190:36663)
0808 04:45:59.448924 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:59.448931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:59.448951 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:45:59.449119 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:59.449123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:59.449187 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:59.449300 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:59.449760 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.450701 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.451888 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.452319 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.452441 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:59.455524 (+ 3083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:48933: BlockingRecv error: recv got EOF from 127.22.2.129:48933 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":118,"threads_started":1}
Aug 08 04:45:59 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:59 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:59.648023 3612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:59.642209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:42123)
0808 04:45:59.642515 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:59.642528 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:59.642620 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:59.642883 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:59.642885 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:59.643067 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:59.643184 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:59.643189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.643906 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.643909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:59.644599 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:59.644610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.645166 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.645171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:59.645275 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:59.645926 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:59.645941 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:59.647904 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
W20260808 04:45:59.648219 3611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:59.642367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:42123 (local address 127.22.2.190:36663)
0808 04:45:59.642549 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:59.642553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:59.642572 (+ 19us) server_negotiation.cc:407] Connection header received
0808 04:45:59.642711 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:59.642713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:59.642758 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:59.642830 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:59.643295 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.643815 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.644720 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.645038 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.645180 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:59.648049 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:42123: BlockingRecv error: recv got EOF from 127.22.2.130:42123 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 04:45:59 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:45:59 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:45:59.856442 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:45:59.850278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:47623)
0808 04:45:59.850540 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 04:45:59.850554 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:45:59.850655 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:45:59.851009 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:45:59.851011 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:45:59.851264 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:45:59.851382 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:59.851388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.852190 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.852192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:59.852799 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:45:59.852814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.853272 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.853275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:45:59.853349 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:59.853929 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:45:59.853942 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:45:59.856317 (+ 2375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
W20260808 04:45:59.856640 3611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:45:59.850296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:47623 (local address 127.22.2.190:36663)
0808 04:45:59.850435 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 04:45:59.850440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:45:59.850629 (+ 189us) server_negotiation.cc:407] Connection header received
0808 04:45:59.850839 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:45:59.850841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:45:59.850890 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:45:59.850990 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:45:59.851494 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.852091 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.852917 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:45:59.853139 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:45:59.853259 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:45:59.856473 (+ 3214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:47623: BlockingRecv error: recv got EOF from 127.22.2.131:47623 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 04:46:00 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:00 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:00.462347 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:00.456448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:34061)
0808 04:46:00.456777 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:00.456790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:00.456885 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:00.457134 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:00.457136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:00.457319 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:00.457417 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:00.457422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.458127 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.458129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:00.459027 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:00.459042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.459684 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.459691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:00.459806 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:00.460357 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:00.460369 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:00.462229 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":120,"threads_started":1}
W20260808 04:46:00.462518 3616 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:00.456505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34061 (local address 127.22.2.190:36663)
0808 04:46:00.456830 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:00.456834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:00.456854 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:46:00.456978 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:00.456980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:00.457024 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:00.457100 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:00.457523 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.458034 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.459185 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.459503 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.459618 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:00.462378 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:34061: BlockingRecv error: recv got EOF from 127.22.2.129:34061 (error 108)
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":93,"threads_started":1}
Aug 08 04:46:00 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:00 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:00.660037 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:00.648860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:41495)
0808 04:46:00.649167 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:00.649180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:00.649314 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:00.649595 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:00.649598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:00.649842 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:00.649941 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:00.649948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.650787 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.650793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:00.651416 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:00.651425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.651887 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.651891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:00.651981 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:00.657742 (+ 5761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:00.657755 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:00.659881 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":94,"threads_started":1}
W20260808 04:46:00.660254 3616 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:00.648927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:41495 (local address 127.22.2.190:36663)
0808 04:46:00.649103 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:00.649108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:00.649265 (+ 157us) server_negotiation.cc:407] Connection header received
0808 04:46:00.649426 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:00.649430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:00.649482 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:00.649572 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:00.650062 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.650646 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.651542 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.651773 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.651885 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:00.660051 (+ 8166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:41495: BlockingRecv error: recv got EOF from 127.22.2.130:41495 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 04:46:00 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:00 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:00.863137 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:00.857399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:50623)
0808 04:46:00.857708 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:00.857722 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:00.857815 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:00.858072 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:00.858074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:00.858259 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:00.858373 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:00.858378 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.859244 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.859247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:00.859810 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:00.859819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.860379 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.860382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:00.860453 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:00.861035 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:00.861047 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:00.862993 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":122,"threads_started":1}
W20260808 04:46:00.863350 3616 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:00.857607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:50623 (local address 127.22.2.190:36663)
0808 04:46:00.857748 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:00.857751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:00.857769 (+ 18us) server_negotiation.cc:407] Connection header received
0808 04:46:00.857907 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:00.857909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:00.857955 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:00.858032 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:00.858521 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.859060 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.859961 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:00.860206 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:00.860334 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:00.863201 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:50623: BlockingRecv error: recv got EOF from 127.22.2.131:50623 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 04:46:01 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:01 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:01.468236 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:01.463094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:42301)
0808 04:46:01.463372 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:01.463398 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:01.463490 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:01.463712 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:01.463714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:01.463895 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:01.463989 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:01.463994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.464745 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.464749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:01.465304 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:01.465313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.465724 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.465727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:01.465787 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:01.466289 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:01.466300 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:01.468097 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
W20260808 04:46:01.468410 3622 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:01.463229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:42301 (local address 127.22.2.190:36663)
0808 04:46:01.463483 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:01.463487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:01.463501 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:46:01.463555 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:01.463557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:01.463598 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:01.463673 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:01.464093 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.464647 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.465410 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.465625 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.465700 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:01.468236 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:42301: BlockingRecv error: recv got EOF from 127.22.2.129:42301 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":82,"threads_started":1}
Aug 08 04:46:01 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:01 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:01.666339 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:01.660939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:39791)
0808 04:46:01.661233 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:01.661252 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:01.661363 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:01.661606 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:01.661608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:01.661809 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:01.661907 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:01.661912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.662687 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.662690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:01.663283 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:01.663294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.663706 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.663709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:01.663771 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:01.664265 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:01.664276 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:01.666225 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":104,"threads_started":1}
W20260808 04:46:01.666538 3622 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:01.660992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:39791 (local address 127.22.2.190:36663)
0808 04:46:01.661142 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:01.661146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:01.661331 (+ 185us) server_negotiation.cc:407] Connection header received
0808 04:46:01.661446 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:01.661448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:01.661493 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:01.661566 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:01.662025 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.662555 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.663394 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.663602 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.663676 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:01.666369 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:39791: BlockingRecv error: recv got EOF from 127.22.2.130:39791 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 04:46:01 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:01 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:01.870492 3625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:01.864033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:37515)
0808 04:46:01.864391 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:01.864410 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:01.864589 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:01.864886 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:01.864889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:01.865107 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:01.865211 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:01.865218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.865996 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.866001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:01.866585 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:01.866593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.867267 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.867270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:01.867364 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:01.867979 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:01.867995 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:01.870334 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":99,"threads_started":1}
W20260808 04:46:01.870565 3622 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:01.864182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:37515 (local address 127.22.2.190:36663)
0808 04:46:01.864356 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:01.864361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:01.864497 (+ 136us) server_negotiation.cc:407] Connection header received
0808 04:46:01.864710 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:01.864714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:01.864772 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:01.864857 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:01.865338 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.865884 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.866727 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:01.867080 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:01.867265 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:01.870471 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:37515: BlockingRecv error: recv got EOF from 127.22.2.131:37515 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 04:46:02 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:02 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:02.475054 3626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:02.469028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:42111)
0808 04:46:02.469358 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:02.469373 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:02.469473 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:02.469766 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:02.469769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:02.469961 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:02.470059 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:02.470065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.470851 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.470853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:02.471494 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:02.471503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.471928 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.471931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:02.471997 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:02.472562 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:02.472575 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:02.474934 (+ 2359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":89,"threads_started":1}
W20260808 04:46:02.475234 3627 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:02.469157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:42111 (local address 127.22.2.190:36663)
0808 04:46:02.469408 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:02.469412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:02.469426 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:46:02.469571 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:02.469573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:02.469639 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:02.469722 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:02.470204 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.470746 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.471603 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.471822 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.471898 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:02.475064 (+ 3166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:42111: BlockingRecv error: recv got EOF from 127.22.2.129:42111 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":62,"threads_started":1}
Aug 08 04:46:02 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:02 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:02.673295 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:02.667115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:47795)
0808 04:46:02.667526 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:02.667544 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:02.667702 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:02.668001 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:02.668004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:02.668270 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:02.668370 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:02.668376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.669376 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.669382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:02.670138 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:02.670148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.670569 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.670571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:02.670633 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:02.671218 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:02.671231 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:02.673179 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":114,"threads_started":1}
W20260808 04:46:02.673478 3627 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:02.667258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:47795 (local address 127.22.2.190:36663)
0808 04:46:02.667501 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:02.667506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:02.667635 (+ 129us) server_negotiation.cc:407] Connection header received
0808 04:46:02.667811 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:02.667815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:02.667884 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:02.667974 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:02.668495 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.669225 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.670249 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.670463 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.670546 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:02.673322 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:47795: BlockingRecv error: recv got EOF from 127.22.2.130:47795 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 08 04:46:02 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:02 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:02.876938 3630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:02.871408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:44697)
0808 04:46:02.871670 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:02.871684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:02.871789 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:02.872078 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:02.872081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:02.872351 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:02.872462 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:02.872469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.873258 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.873261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:02.873817 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:02.873825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.874264 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.874267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:02.874329 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:02.874825 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:02.874836 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:02.876821 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
W20260808 04:46:02.877149 3627 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:02.871524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:44697 (local address 127.22.2.190:36663)
0808 04:46:02.871707 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:02.871710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:02.871741 (+ 31us) server_negotiation.cc:407] Connection header received
0808 04:46:02.871881 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:02.871883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:02.871930 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:02.871997 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:02.872594 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.873117 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.873923 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:02.874137 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:02.874220 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:02.876968 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:44697: BlockingRecv error: recv got EOF from 127.22.2.131:44697 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 04:46:03 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:03 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:03.482157 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:03.475957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:47829)
0808 04:46:03.476257 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:03.476276 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:03.476385 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:03.476693 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:03.476699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:03.476999 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:03.477153 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:03.477161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.477967 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.477971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:03.478758 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:03.478770 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.479282 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.479285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:03.479356 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:03.479997 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:03.480011 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:03.482037 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":74,"threads_started":1}
W20260808 04:46:03.482311 3633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:03.476102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:47829 (local address 127.22.2.190:36663)
0808 04:46:03.476348 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:03.476351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:03.476365 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:46:03.476461 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:03.476463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:03.476505 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:03.476652 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:03.477279 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.477825 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.478927 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.479170 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.479253 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:03.482172 (+ 2919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:47829: BlockingRecv error: recv got EOF from 127.22.2.129:47829 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
Aug 08 04:46:03 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:03 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:03.679831 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:03.674068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:33653)
0808 04:46:03.674389 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:03.674405 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:03.674523 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:03.674819 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:03.674822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:03.675036 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:03.675132 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:03.675138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.675928 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.675934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:03.676552 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:03.676562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.677035 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.677038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:03.677126 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:03.677636 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:03.677648 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:03.679668 (+ 2020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":110,"threads_started":1}
W20260808 04:46:03.679926 3633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:03.674272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:33653 (local address 127.22.2.190:36663)
0808 04:46:03.674433 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:03.674440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:03.674460 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:46:03.674636 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:03.674640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:03.674695 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:03.674782 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:03.675260 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.675810 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.676676 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.676912 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.677028 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:03.679798 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:33653: BlockingRecv error: recv got EOF from 127.22.2.130:33653 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 04:46:03 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:03 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:03.884122 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:03.877880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:45435)
0808 04:46:03.878161 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:03.878175 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:03.878277 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:03.878557 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:03.878560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:03.878758 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:03.878851 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:03.878857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.879609 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.879613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:03.880255 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:03.880265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.880811 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.880815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:03.880897 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:03.881550 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:03.881568 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:03.883925 (+ 2357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
W20260808 04:46:03.884194 3633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:03.878023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:45435 (local address 127.22.2.190:36663)
0808 04:46:03.878174 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:03.878178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:03.878237 (+ 59us) server_negotiation.cc:407] Connection header received
0808 04:46:03.878366 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:03.878368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:03.878414 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:03.878511 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:03.878943 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.879464 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.880375 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:03.880666 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:03.880752 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:03.884050 (+ 3298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:45435: BlockingRecv error: recv got EOF from 127.22.2.131:45435 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 04:46:04 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:04 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:04.488952 3639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:04.482980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:52473)
0808 04:46:04.483261 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:04.483278 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:04.483403 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:04.483732 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:04.483735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:04.484001 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:04.484142 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:04.484155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.485107 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.485110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:04.485681 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:04.485690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.486118 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.486121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:04.486184 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:04.486742 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:04.486754 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:04.488795 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":97,"threads_started":1}
W20260808 04:46:04.489138 3640 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:04.483076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:52473 (local address 127.22.2.190:36663)
0808 04:46:04.483325 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:04.483328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:04.483344 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:46:04.483509 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:04.483513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:04.483573 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:04.483727 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:04.484283 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.484998 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.485793 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.486015 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.486097 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:04.488952 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:52473: BlockingRecv error: recv got EOF from 127.22.2.129:52473 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":83,"threads_started":1}
Aug 08 04:46:04 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:04 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:04.686064 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:04.680642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:51363)
0808 04:46:04.680893 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:04.680909 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:04.681018 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:04.681258 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:04.681261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:04.681447 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:04.681545 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:04.681550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.682266 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.682269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:04.682826 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:04.682834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.683248 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.683251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:04.683387 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:04.683986 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:04.683999 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:04.685946 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
W20260808 04:46:04.686203 3640 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:04.680661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:51363 (local address 127.22.2.190:36663)
0808 04:46:04.680824 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:04.680829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:04.681009 (+ 180us) server_negotiation.cc:407] Connection header received
0808 04:46:04.681091 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:04.681093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:04.681142 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:04.681217 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:04.681646 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.682161 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.682927 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.683152 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.683225 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:04.686081 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:51363: BlockingRecv error: recv got EOF from 127.22.2.130:51363 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 04:46:04 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:04 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:04.891052 3644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:04.885004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:55673)
0808 04:46:04.885275 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:04.885289 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:04.885396 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:04.885646 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:04.885648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:04.885855 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:04.885951 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:04.885956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.886819 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.886822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:04.887361 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:04.887368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.887987 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.887989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:04.888051 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:04.888678 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:04.888713 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:04.890934 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
W20260808 04:46:04.891305 3640 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:04.885096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:55673 (local address 127.22.2.190:36663)
0808 04:46:04.885305 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:04.885309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:04.885361 (+ 52us) server_negotiation.cc:407] Connection header received
0808 04:46:04.885487 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:04.885489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:04.885539 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:04.885611 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:04.886087 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.886700 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.887477 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:04.887810 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:04.887990 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:04.891145 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:55673: BlockingRecv error: recv got EOF from 127.22.2.131:55673 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 08 04:46:05 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:05 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:05.495671 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:05.489793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:49133)
0808 04:46:05.490164 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:05.490178 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:05.490292 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:05.490545 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:05.490548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:05.490742 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:05.490838 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:05.490844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.491571 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.491574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:05.492365 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:05.492377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.492868 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.492871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:05.492940 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:05.493503 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:05.493515 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:05.495497 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":90,"threads_started":1}
W20260808 04:46:05.495752 3647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:05.489873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:49133 (local address 127.22.2.190:36663)
0808 04:46:05.490180 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:05.490183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:05.490250 (+ 67us) server_negotiation.cc:407] Connection header received
0808 04:46:05.490381 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:05.490383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:05.490429 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:05.490506 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:05.490936 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.491461 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.492507 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.492754 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.492846 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:05.495627 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:49133: BlockingRecv error: recv got EOF from 127.22.2.129:49133 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":66,"threads_started":1}
Aug 08 04:46:05 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:05 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:05.699043 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:05.686959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:53137)
0808 04:46:05.687312 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:05.687337 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:05.687480 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:05.687769 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:05.687774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:05.687981 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:05.688081 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:05.688087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.689124 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.689128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:05.689953 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:05.689965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.690490 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.690494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:05.690568 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:05.696481 (+ 5913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:05.696496 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:05.698861 (+ 2365us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":135,"threads_started":1}
W20260808 04:46:05.699101 3647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:05.687081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53137 (local address 127.22.2.190:36663)
0808 04:46:05.687314 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:05.687320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:05.687423 (+ 103us) server_negotiation.cc:407] Connection header received
0808 04:46:05.687565 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:05.687568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:05.687615 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:05.687691 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:05.688216 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.688986 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.690088 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.690357 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.690451 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:05.698992 (+ 8541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:53137: BlockingRecv error: recv got EOF from 127.22.2.130:53137 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 08 04:46:05 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:05 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:05.897945 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:05.891850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:54547)
0808 04:46:05.892203 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:05.892215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:05.892351 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:05.892669 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:05.892672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:05.892901 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:05.892997 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:05.893003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.893779 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.893784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:05.894351 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:05.894358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.894837 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.894841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:05.894929 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:05.895449 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:05.895460 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:05.897739 (+ 2279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":95,"threads_started":1}
W20260808 04:46:05.898046 3647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:05.891922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:54547 (local address 127.22.2.190:36663)
0808 04:46:05.892091 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:05.892097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:05.892295 (+ 198us) server_negotiation.cc:407] Connection header received
0808 04:46:05.892465 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:05.892468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:05.892545 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:05.892637 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:05.893124 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.893663 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.894477 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:05.894707 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:05.894824 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:05.897896 (+ 3072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:54547: BlockingRecv error: recv got EOF from 127.22.2.131:54547 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 04:46:06 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:06 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:06.502080 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:06.496511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:34803)
0808 04:46:06.496864 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:06.496878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:06.496970 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:06.497246 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:06.497248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:06.497452 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:06.497551 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:06.497557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.498355 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.498358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:06.498900 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:06.498907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.499420 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.499423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:06.499486 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:06.499977 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:06.499989 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:06.501928 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":94,"threads_started":1}
W20260808 04:46:06.502379 3653 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:06.496696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34803 (local address 127.22.2.190:36663)
0808 04:46:06.496963 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:06.496966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:06.496980 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:46:06.497040 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:06.497042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:06.497090 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:06.497204 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:06.497668 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.498241 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.499029 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.499304 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.499402 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:06.502098 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:34803: BlockingRecv error: recv got EOF from 127.22.2.129:34803 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
Aug 08 04:46:06 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:06 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:06.705958 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:06.699847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:35837)
0808 04:46:06.700193 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:06.700209 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:06.700330 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:06.700668 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:06.700671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:06.700869 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:06.700970 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:06.700976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.701940 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.701943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:06.702519 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:06.702527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.702986 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.702989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:06.703050 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:06.703593 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:06.703607 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:06.705764 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":126,"threads_started":1}
W20260808 04:46:06.706058 3653 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:06.699992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:35837 (local address 127.22.2.190:36663)
0808 04:46:06.700158 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:06.700162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:06.700286 (+ 124us) server_negotiation.cc:407] Connection header received
0808 04:46:06.700431 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:06.700434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:06.700485 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:06.700595 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:06.701084 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.701821 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.702629 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.702883 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.702967 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:06.705923 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:35837: BlockingRecv error: recv got EOF from 127.22.2.130:35837 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 04:46:06 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:06 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:06.904333 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:06.898745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:59043)
0808 04:46:06.899057 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:06.899071 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:06.899183 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:06.899433 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:06.899435 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:06.899631 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:06.899729 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:06.899735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.900517 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.900541 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:06.901127 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:06.901138 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.901637 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.901641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:06.901715 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:06.902244 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:06.902257 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:06.904157 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":118,"threads_started":1}
W20260808 04:46:06.904420 3653 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:06.898803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:59043 (local address 127.22.2.190:36663)
0808 04:46:06.899034 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:06.899038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:06.899187 (+ 149us) server_negotiation.cc:407] Connection header received
0808 04:46:06.899269 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:06.899271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:06.899320 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:06.899388 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:06.899835 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.900398 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.901252 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:06.901483 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:06.901571 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:06.904280 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:59043: BlockingRecv error: recv got EOF from 127.22.2.131:59043 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 08 04:46:07 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:07 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:07.508399 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:07.502910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:45027)
0808 04:46:07.503209 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:07.503223 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:07.503324 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:07.503584 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:07.503587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:07.503769 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:07.503866 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:07.503871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.504657 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.504660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:07.505205 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:07.505212 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.505633 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.505636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:07.505694 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:07.506413 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:07.506425 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:07.508271 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":102,"threads_started":1}
W20260808 04:46:07.508605 3660 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:07.502978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:45027 (local address 127.22.2.190:36663)
0808 04:46:07.503259 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:07.503264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:07.503280 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:46:07.503418 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:07.503421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:07.503467 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:07.503545 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:07.503976 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.504562 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.505310 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.505536 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.505614 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:07.508410 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:45027: BlockingRecv error: recv got EOF from 127.22.2.129:45027 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
Aug 08 04:46:07 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:07 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:07.713037 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:07.706822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:60915)
0808 04:46:07.707173 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:07.707187 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:07.707342 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:07.707676 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:07.707679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:07.707928 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:07.708078 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:07.708088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.709007 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.709010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:07.709558 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:07.709566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.710127 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.710134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:07.710259 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:07.710936 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:07.710949 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:07.712919 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":92,"threads_started":1}
W20260808 04:46:07.713176 3660 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:07.706952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:60915 (local address 127.22.2.190:36663)
0808 04:46:07.707134 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:07.707139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:07.707299 (+ 160us) server_negotiation.cc:407] Connection header received
0808 04:46:07.707460 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:07.707464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:07.707526 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:07.707630 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:07.708197 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.708902 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.709656 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.709996 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.710097 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:07.713053 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:60915: BlockingRecv error: recv got EOF from 127.22.2.130:60915 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 04:46:07 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:07 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:07.910890 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:07.905286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:35949)
0808 04:46:07.905599 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:07.905613 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:07.905789 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:07.906082 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:07.906085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:07.906309 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:07.906409 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:07.906415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.907184 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.907190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:07.907763 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:07.907771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.908268 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.908272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:07.908359 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:07.908917 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:07.908928 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:07.910773 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":94,"threads_started":1}
W20260808 04:46:07.911078 3660 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:07.905355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35949 (local address 127.22.2.190:36663)
0808 04:46:07.905487 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:07.905492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:07.905723 (+ 231us) server_negotiation.cc:407] Connection header received
0808 04:46:07.905904 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:07.905908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:07.905965 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:07.906054 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:07.906536 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.907071 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.907906 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:07.908137 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:07.908267 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:07.910913 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:35949: BlockingRecv error: recv got EOF from 127.22.2.131:35949 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 04:46:08 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:08 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:08.515285 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:08.509280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:56775)
0808 04:46:08.509616 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:08.509631 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:08.509739 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:08.510011 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:08.510013 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:08.510238 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:08.510386 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:08.510394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.511378 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.511384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:08.512006 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:08.512015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.512495 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.512499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:08.512620 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:08.513131 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:08.513142 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:08.515089 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":74,"threads_started":1}
W20260808 04:46:08.515350 3666 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:08.509363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56775 (local address 127.22.2.190:36663)
0808 04:46:08.509675 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:08.509679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:08.509694 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:46:08.509832 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:08.509834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:08.509877 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:08.509956 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:08.510513 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.511263 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.512133 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.512375 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.512493 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:08.515232 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:56775: BlockingRecv error: recv got EOF from 127.22.2.129:56775 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
Aug 08 04:46:08 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:08 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:08.719844 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:08.713945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:36611)
0808 04:46:08.714258 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:08.714272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:08.714352 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:08.714627 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:08.714630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:08.714884 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:08.714991 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:08.714997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.715786 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.715789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:08.716350 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:08.716357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.716788 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.716791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:08.716869 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:08.717489 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:08.717506 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:08.719728 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":91,"threads_started":1}
W20260808 04:46:08.719998 3666 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:08.714038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:36611 (local address 127.22.2.190:36663)
0808 04:46:08.714295 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:08.714298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:08.714335 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:46:08.714441 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:08.714443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:08.714486 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:08.714603 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:08.715109 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.715685 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.716451 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.716695 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.716772 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:08.719871 (+ 3099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:36611: BlockingRecv error: recv got EOF from 127.22.2.130:36611 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 08 04:46:08 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:08 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:08.917654 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:08.911685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:34641)
0808 04:46:08.911989 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:08.912003 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:08.912083 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:08.912376 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:08.912378 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:08.912589 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:08.912686 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:08.912692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.913492 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.913494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:08.914053 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:08.914061 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.914723 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.914728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:08.914814 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:08.915464 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:08.915480 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:08.917484 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
W20260808 04:46:08.917721 3666 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:08.911894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:34641 (local address 127.22.2.190:36663)
0808 04:46:08.912049 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:08.912054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:08.912069 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:46:08.912208 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:08.912211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:08.912259 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:08.912345 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:08.912786 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.913392 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.914272 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:08.914581 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:08.914716 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:08.917613 (+ 2897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:34641: BlockingRecv error: recv got EOF from 127.22.2.131:34641 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 04:46:09 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:09 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:09.521821 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:09.516266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:35007)
0808 04:46:09.516625 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:09.516639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:09.516730 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:09.516987 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:09.516989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:09.517170 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:09.517269 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:09.517274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.518071 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.518074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:09.518639 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:09.518648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.519061 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.519063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:09.519126 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:09.519675 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:09.519687 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:09.521672 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
W20260808 04:46:09.521937 3672 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:09.516436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:35007 (local address 127.22.2.190:36663)
0808 04:46:09.516701 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:09.516704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:09.516718 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:46:09.516815 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:09.516817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:09.516863 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:09.516941 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:09.517418 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.517967 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.518740 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.518960 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.519035 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:09.521806 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:35007: BlockingRecv error: recv got EOF from 127.22.2.129:35007 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":105,"threads_started":1}
Aug 08 04:46:09 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:09 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:09.726691 3673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:09.720732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:32847)
0808 04:46:09.721096 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:09.721121 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:09.721286 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:09.721566 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:09.721569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:09.721792 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:09.721887 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:09.721893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.722810 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.722816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:09.723397 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:09.723406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.723912 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.723915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:09.724003 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:09.724626 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:09.724639 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:09.726520 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":147,"threads_started":1}
W20260808 04:46:09.726752 3672 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:09.720893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:32847 (local address 127.22.2.190:36663)
0808 04:46:09.721084 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:09.721088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:09.721218 (+ 130us) server_negotiation.cc:407] Connection header received
0808 04:46:09.721396 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:09.721399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:09.721451 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:09.721542 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:09.722012 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.722705 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.723540 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.723790 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.723910 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:09.726647 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:32847: BlockingRecv error: recv got EOF from 127.22.2.130:32847 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 04:46:09 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:09 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:09.924957 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:09.918595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:45509)
0808 04:46:09.918921 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:09.918934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:09.919079 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:09.919380 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:09.919384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:09.919601 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:09.919711 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:09.919717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.920501 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.920506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:09.921131 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:09.921140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.921641 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.921645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:09.921736 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:09.922317 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:09.922337 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:09.924767 (+ 2430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":118,"threads_started":1}
W20260808 04:46:09.925031 3672 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:09.918667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:45509 (local address 127.22.2.190:36663)
0808 04:46:09.918846 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:09.918850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:09.919013 (+ 163us) server_negotiation.cc:407] Connection header received
0808 04:46:09.919184 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:09.919188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:09.919253 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:09.919348 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:09.919837 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.920383 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.921292 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:09.921514 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:09.921626 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:09.924909 (+ 3283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:45509: BlockingRecv error: recv got EOF from 127.22.2.131:45509 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 04:46:10 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:10 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:10.528954 3675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:10.522733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:59007)
0808 04:46:10.523058 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:10.523072 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:10.523172 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:10.523441 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:10.523443 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:10.523631 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:10.523732 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:10.523738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.524591 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.524595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:10.525594 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:10.525606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.526126 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.526130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:10.526228 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:10.526766 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:10.526778 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:10.528775 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":133,"threads_started":1}
W20260808 04:46:10.529034 3676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:10.522834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:59007 (local address 127.22.2.190:36663)
0808 04:46:10.523084 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:10.523088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:10.523124 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:46:10.523260 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:10.523263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:10.523309 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:10.523390 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:10.523831 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.524434 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.525701 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.525988 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.526128 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:10.528905 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:59007: BlockingRecv error: recv got EOF from 127.22.2.129:59007 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
Aug 08 04:46:10 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:10 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:10.739406 3677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:10.727543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:36469)
0808 04:46:10.727915 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:10.727933 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:10.728066 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:10.728415 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:10.728418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:10.728720 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:10.728884 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:10.728894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.729714 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.729716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:10.730278 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:10.730285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.730740 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.730743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:10.730803 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:10.736978 (+ 6175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:10.736991 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:10.739237 (+ 2246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":104,"threads_started":1}
W20260808 04:46:10.739691 3676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:10.727627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:36469 (local address 127.22.2.190:36663)
0808 04:46:10.727812 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:10.727817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:10.728043 (+ 226us) server_negotiation.cc:407] Connection header received
0808 04:46:10.728183 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:10.728187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:10.728264 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:10.728373 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:10.729013 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.729611 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.730387 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.730635 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.730731 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:10.739393 (+ 8662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:36469: BlockingRecv error: recv got EOF from 127.22.2.130:36469 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 04:46:10 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:10 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:10.932570 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:10.925992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:46687)
0808 04:46:10.926349 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:10.926362 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:10.926506 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:10.926842 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:10.926844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:10.927028 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:10.927132 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:10.927138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.928093 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.928097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:10.928858 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:10.928871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.929476 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.929481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:10.929575 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:10.930224 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:10.930238 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:10.932354 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":118,"threads_started":1}
W20260808 04:46:10.932664 3676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:10.926080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:46687 (local address 127.22.2.190:36663)
0808 04:46:10.926256 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:10.926261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:10.926443 (+ 182us) server_negotiation.cc:407] Connection header received
0808 04:46:10.926617 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:10.926622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:10.926704 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:10.926791 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:10.927280 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.927931 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.928995 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:10.929344 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:10.929473 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:10.932490 (+ 3017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:46687: BlockingRecv error: recv got EOF from 127.22.2.131:46687 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 04:46:11 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:11 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:11.535358 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:11.529811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:39931)
0808 04:46:11.530206 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:11.530222 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:11.530333 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:11.530628 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:11.530631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:11.530823 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:11.530928 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:11.530934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.531698 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.531701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:11.532299 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:11.532308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.532745 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.532747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:11.532812 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:11.533332 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:11.533344 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:11.535205 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
W20260808 04:46:11.535686 3682 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:11.529920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:39931 (local address 127.22.2.190:36663)
0808 04:46:11.530180 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:11.530184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:11.530304 (+ 120us) server_negotiation.cc:407] Connection header received
0808 04:46:11.530418 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:11.530421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:11.530466 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:11.530548 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:11.531040 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.531578 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.532407 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.532647 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.532723 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:11.535424 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:39931: BlockingRecv error: recv got EOF from 127.22.2.129:39931 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":63,"threads_started":1}
Aug 08 04:46:11 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:11 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:11.746584 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:11.740245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:54257)
0808 04:46:11.740737 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:11.740756 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:11.740883 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:11.741201 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:11.741205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:11.741438 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:11.741541 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:11.741547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.742433 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.742440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:11.743065 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:11.743074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.743596 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.743601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:11.743694 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:11.744354 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:11.744367 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:11.746379 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":230,"threads_started":1}
W20260808 04:46:11.746657 3682 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:11.740559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:54257 (local address 127.22.2.190:36663)
0808 04:46:11.740741 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:11.740749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:11.740820 (+ 71us) server_negotiation.cc:407] Connection header received
0808 04:46:11.741006 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:11.741010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:11.741073 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:11.741174 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:11.741691 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.742306 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.743208 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.743470 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.743603 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:11.746520 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:54257: BlockingRecv error: recv got EOF from 127.22.2.130:54257 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 04:46:11 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:11 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:11.938890 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:11.933411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:52123)
0808 04:46:11.933715 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:11.933729 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:11.933853 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:11.934114 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:11.934116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:11.934297 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:11.934397 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:11.934403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.935147 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.935149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:11.935687 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:11.935694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.936202 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.936204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:11.936264 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:11.936905 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:11.936917 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:11.938754 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":124,"threads_started":1}
W20260808 04:46:11.939060 3682 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:11.933498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:52123 (local address 127.22.2.190:36663)
0808 04:46:11.933638 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:11.933642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:11.933825 (+ 183us) server_negotiation.cc:407] Connection header received
0808 04:46:11.933946 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:11.933948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:11.933997 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:11.934073 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:11.934507 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.935049 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.935815 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:11.936038 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:11.936185 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:11.938899 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:52123: BlockingRecv error: recv got EOF from 127.22.2.131:52123 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 04:46:12 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:12 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:12.541952 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:12.536166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:39013)
0808 04:46:12.536461 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:12.536476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:12.536635 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:12.536974 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:12.536977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:12.537191 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:12.537297 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:12.537304 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.538099 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.538102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:12.538726 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:12.538736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.539202 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.539213 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:12.539283 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:12.539781 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:12.539793 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:12.541823 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":107,"threads_started":1}
W20260808 04:46:12.542133 3689 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:12.536362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:39013 (local address 127.22.2.190:36663)
0808 04:46:12.536715 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:12.536718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:12.536731 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:46:12.536767 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:12.536770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:12.536831 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:12.536915 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:12.537420 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.537971 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.538851 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.539069 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.539146 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:12.541988 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:39013: BlockingRecv error: recv got EOF from 127.22.2.129:39013 (error 108)
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":136,"threads_started":1}
Aug 08 04:46:12 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:12 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:12.754994 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:12.747625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:40593)
0808 04:46:12.747954 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:12.747969 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:12.748081 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:12.748389 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:12.748392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:12.748666 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:12.748784 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:12.748790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.749599 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.749603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:12.750438 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:12.750448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.750939 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.750943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:12.751059 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:12.751810 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:12.751831 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:12.754828 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":128,"threads_started":1}
W20260808 04:46:12.755084 3689 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:12.747714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:40593 (local address 127.22.2.190:36663)
0808 04:46:12.747913 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:12.747918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:12.748027 (+ 109us) server_negotiation.cc:407] Connection header received
0808 04:46:12.748214 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:12.748216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:12.748262 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:12.748330 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:12.748891 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.749487 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.750545 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.750821 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.750913 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:12.754975 (+ 4062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:40593: BlockingRecv error: recv got EOF from 127.22.2.130:40593 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 04:46:12 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:12 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:12.945509 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:12.939674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.131:42047)
0808 04:46:12.939955 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:12.939968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:12.940072 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:12.940334 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:12.940336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:12.940545 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:12.940641 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:12.940647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.941381 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.941385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:12.942224 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:12.942236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.942877 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.942880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:12.942947 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:12.943496 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:12.943507 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:12.945392 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":73,"threads_started":1}
W20260808 04:46:12.945724 3689 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:12.939754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:42047 (local address 127.22.2.190:36663)
0808 04:46:12.939970 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:12.939974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:12.940040 (+ 66us) server_negotiation.cc:407] Connection header received
0808 04:46:12.940152 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:12.940154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:12.940209 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:12.940280 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:12.940745 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.941264 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.942412 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:12.942744 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:12.942834 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:12.945542 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:42047: BlockingRecv error: recv got EOF from 127.22.2.131:42047 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 08 04:46:13 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:13 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:13.548794 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:13.542907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.129:41023)
0808 04:46:13.543242 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:13.543256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:13.543346 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:13.543615 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:13.543617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:13.543815 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:13.543919 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:13.543924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:13.544674 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:13.544677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:13.545268 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:13.545278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:13.545781 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:13.545784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:13.545878 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:13.546539 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:13.546554 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:13.548677 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":110,"threads_started":1}
W20260808 04:46:13.548997 3695 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:13.543038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:41023 (local address 127.22.2.190:36663)
0808 04:46:13.543289 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:13.543293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:13.543309 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:46:13.543440 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:13.543442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:13.543487 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:13.543573 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:13.544025 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:13.544564 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:13.545386 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:13.545645 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:13.545766 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:13.548822 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:41023: BlockingRecv error: recv got EOF from 127.22.2.129:41023 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
Aug 08 04:46:13 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:46:13 dist-test-slave-04bb krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:46:13.762739 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:13.757058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36663 (local address 127.22.2.130:56027)
0808 04:46:13.757411 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:13.757429 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:13.757567 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:13.757876 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:13.757886 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:13.758106 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:13.758198 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:13.758204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:13.759018 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:13.759023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:13.759592 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:13.759600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:13.760075 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:13.760078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:13.760170 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:13.760725 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:13.760738 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:13.762569 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:36663: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":90,"threads_started":1}
W20260808 04:46:13.762830 3695 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:13.757246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:56027 (local address 127.22.2.190:36663)
0808 04:46:13.757414 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:13.757421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:13.757513 (+ 92us) server_negotiation.cc:407] Connection header received
0808 04:46:13.757676 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:13.757680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:13.757749 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:13.757851 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:13.758347 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:13.758905 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:13.759720 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:13.759959 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:13.760071 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:13.762698 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:56027: BlockingRecv error: recv got EOF from 127.22.2.130:56027 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 04:46:13.836659 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 3031
I20260808 04:46:13.860404 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 3243
I20260808 04:46:13.870872 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 3427
I20260808 04:46:13.887508 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 2937
2026-08-08T04:46:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18365 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T04:46:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:46:13Z Disabled control of system clock
I20260808 04:46:13.926537 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40261
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 04:46:14.039494 3701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:14.039741 3701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:14.039775 3701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:14.043733 3701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:46:14.043783 3701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:14.043799 3701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:14.043815 3701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:46:14.043831 3701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:46:14.048326 3701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:14.049347 3701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:14.050633 3701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:14.059696 3706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:14.059917 3701 server_base.cc:1034] running on GCE node
W20260808 04:46:14.059671 3710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:14.059696 3708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:14.060742 3701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:14.061929 3701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:14.063128 3701 hybrid_clock.cc:648] HybridClock initialized: now 1786164374063100 us; error 44 us; skew 500 ppm
I20260808 04:46:14.065220 3701 webserver.cc:460] Webserver started at http://127.22.2.190:42515/ using document root <none> and password file <none>
I20260808 04:46:14.065836 3701 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:14.065902 3701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:14.066162 3701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:14.068262 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "92668416c74c42079a11d753c2d893fd"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "d2d195369518ca44fa74f5ea2b96e39e"
server_key_iv: "119198b87c870d5c91dd584c9015e76c"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.068878 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "92668416c74c42079a11d753c2d893fd"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "d2d195369518ca44fa74f5ea2b96e39e"
server_key_iv: "119198b87c870d5c91dd584c9015e76c"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.072649 3701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:46:14.075109 3718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.076395 3701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:46:14.076591 3701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "92668416c74c42079a11d753c2d893fd"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "d2d195369518ca44fa74f5ea2b96e39e"
server_key_iv: "119198b87c870d5c91dd584c9015e76c"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.076723 3701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:14.098572 3701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:14.099364 3701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:14.099579 3701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:14.107275 3701 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:40261
I20260808 04:46:14.107280 3791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:40261 every 8 connection(s)
I20260808 04:46:14.108279 3701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:46:14.111701 3793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:46:14.112776 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 3701
I20260808 04:46:14.112910 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:46:14.113226 22538 external_mini_cluster.cc:1428] Setting key f8fbbf1cbf32e06ed05edfc001bcc9b4
I20260808 04:46:14.118184 3793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd: Bootstrap starting.
I20260808 04:46:14.120500 3793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd: Neither blocks nor log segments found. Creating new log.
I20260808 04:46:14.121331 3793 log.cc:824] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd: Log is configured to *not* fsync() on all Append() calls
I20260808 04:46:14.122584 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:14.114707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49856 (local address 127.22.2.190:40261)
0808 04:46:14.115403 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:14.115419 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:14.115455 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:46:14.116101 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:14.116119 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:14.116595 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:14.116947 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:14.117400 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.118367 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.119255 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.119729 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.119903 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:14.120870 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:14.120876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:14.120888 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:46:14.121275 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:14.121332 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:46:14.121353 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:14.121465 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:14.121602 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 04:46:14.121897 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":143,"threads_started":1}
I20260808 04:46:14.123404 3793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd: No bootstrap required, opened a new log
I20260808 04:46:14.127136 3793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92668416c74c42079a11d753c2d893fd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40261 } }
I20260808 04:46:14.127346 3793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:46:14.127381 3793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92668416c74c42079a11d753c2d893fd, State: Initialized, Role: FOLLOWER
I20260808 04:46:14.127784 3793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92668416c74c42079a11d753c2d893fd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40261 } }
I20260808 04:46:14.127928 3793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:46:14.127969 3793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:46:14.128036 3793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:46:14.128934 3793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92668416c74c42079a11d753c2d893fd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40261 } }
I20260808 04:46:14.129294 3793 leader_election.cc:302] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92668416c74c42079a11d753c2d893fd; no voters:
I20260808 04:46:14.129689 3793 leader_election.cc:288] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:46:14.129772 3800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:46:14.129987 3800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [term 1 LEADER]: Becoming Leader. State: Replica: 92668416c74c42079a11d753c2d893fd, State: Running, Role: LEADER
I20260808 04:46:14.130299 3800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92668416c74c42079a11d753c2d893fd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40261 } }
I20260808 04:46:14.131032 3793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:46:14.132107 3801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92668416c74c42079a11d753c2d893fd. Latest consensus state: current_term: 1 leader_uuid: "92668416c74c42079a11d753c2d893fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92668416c74c42079a11d753c2d893fd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40261 } } }
I20260808 04:46:14.132257 3801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [sys.catalog]: This master's current role is: LEADER
I20260808 04:46:14.132575 3804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92668416c74c42079a11d753c2d893fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92668416c74c42079a11d753c2d893fd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40261 } } }
I20260808 04:46:14.132670 3804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd [sys.catalog]: This master's current role is: LEADER
I20260808 04:46:14.132807 3815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:46:14.136164 3815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:46:14.142827 3815 catalog_manager.cc:1347] Generated new cluster ID: 1b4dbef0c62a460fa0b8908688595b00
I20260808 04:46:14.142913 3815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:46:14.161808 3815 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:46:14.162714 3815 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:46:14.170718 3815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92668416c74c42079a11d753c2d893fd: Generated new TSK 0
I20260808 04:46:14.171425 3815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:46:14.180189 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40261
--builtin_ntp_servers=127.22.2.148:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 04:46:14.300925 3826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:14.301196 3826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:14.301236 3826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:14.305207 3826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:14.305258 3826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:14.305368 3826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:46:14.310079 3826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:14.311136 3826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:14.312574 3826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:14.322100 3834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:14.322101 3839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:14.322130 3835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:14.322363 3826 server_base.cc:1034] running on GCE node
I20260808 04:46:14.322774 3826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:14.323415 3826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:14.324599 3826 hybrid_clock.cc:648] HybridClock initialized: now 1786164374324575 us; error 34 us; skew 500 ppm
I20260808 04:46:14.326603 3826 webserver.cc:460] Webserver started at http://127.22.2.129:36005/ using document root <none> and password file <none>
I20260808 04:46:14.327339 3826 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:14.327437 3826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:14.327680 3826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:14.329821 3826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "cd89903304b34e63b0cf2e513bd26783"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "eb6346f436326c0e5f168fb639a9fca7"
server_key_iv: "79157b233b7591cb7e463b4c38935ab8"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.330404 3826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "cd89903304b34e63b0cf2e513bd26783"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "eb6346f436326c0e5f168fb639a9fca7"
server_key_iv: "79157b233b7591cb7e463b4c38935ab8"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.334460 3826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:46:14.337040 3848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.338291 3826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:46:14.338429 3826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "cd89903304b34e63b0cf2e513bd26783"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "eb6346f436326c0e5f168fb639a9fca7"
server_key_iv: "79157b233b7591cb7e463b4c38935ab8"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.338552 3826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:14.365556 3826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:14.366267 3826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:14.366469 3826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:14.367075 3826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:14.368098 3826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:14.368172 3826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.368247 3826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:14.368292 3826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.378000 3826 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:32837
I20260808 04:46:14.378082 4014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:32837 every 8 connection(s)
I20260808 04:46:14.378958 3826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:46:14.388350 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 3826
I20260808 04:46:14.388487 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:46:14.388669 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:14.381119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:33227 (local address 127.22.2.190:40261)
0808 04:46:14.381261 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:14.381270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:14.382355 (+ 1085us) server_negotiation.cc:407] Connection header received
0808 04:46:14.383732 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:14.383736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:14.383790 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:14.383875 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:14.384860 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.385436 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.386538 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.386765 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.386850 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:14.387991 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:14.387997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:14.388004 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:46:14.388032 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:14.388060 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:46:14.388074 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:14.388234 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:14.388376 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 04:46:14.388458 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:46:14.388911 22538 external_mini_cluster.cc:1428] Setting key c1496cde1c184624753ca59c1383d68d
I20260808 04:46:14.389642 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:14.381521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40261 (local address 127.22.2.129:33227)
0808 04:46:14.382189 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:14.382230 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:14.383502 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:14.384042 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:14.384049 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:14.384131 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:14.384700 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:14.384715 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.385549 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.385552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:14.386423 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:14.386432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.386898 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.386900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:14.387042 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:14.387671 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:46:14.387673 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:14.387712 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:46:14.387774 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:14.388223 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:14.388230 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:14.388234 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:46:14.388338 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 04:46:14.388664 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":115,"threads_started":1}
I20260808 04:46:14.391009 4015 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40261
I20260808 04:46:14.391196 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40261
--builtin_ntp_servers=127.22.2.148:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 04:46:14.391537 4015 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:14.392078 4015 heartbeater.cc:511] Master 127.22.2.190:40261 requested a full tablet report, sending...
I20260808 04:46:14.393728 3738 ts_manager.cc:194] Registered new tserver with Master: cd89903304b34e63b0cf2e513bd26783 (127.22.2.129:32837)
I20260808 04:46:14.394876 3738 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.129:33227
W20260808 04:46:14.504508 4018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:14.504884 4018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:14.504961 4018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:14.508962 4018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:14.509011 4018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:14.509102 4018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:46:14.513816 4018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:14.514854 4018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:14.516294 4018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:14.526005 4031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:14.525991 4025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:14.526062 4026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:14.526285 4018 server_base.cc:1034] running on GCE node
I20260808 04:46:14.526700 4018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:14.527392 4018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:14.528594 4018 hybrid_clock.cc:648] HybridClock initialized: now 1786164374528564 us; error 28 us; skew 500 ppm
I20260808 04:46:14.530412 4018 webserver.cc:460] Webserver started at http://127.22.2.130:34985/ using document root <none> and password file <none>
I20260808 04:46:14.531067 4018 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:14.531133 4018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:14.531299 4018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:14.533869 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "852563c4bc5e4d8ab144dc7a537c963b"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "b64e31464dd6f081bc6a5170239787d1"
server_key_iv: "0bd3a18965797bc53648d4c44967500a"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.534624 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "852563c4bc5e4d8ab144dc7a537c963b"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "b64e31464dd6f081bc6a5170239787d1"
server_key_iv: "0bd3a18965797bc53648d4c44967500a"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.538791 4018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:46:14.541865 4039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.543358 4018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:46:14.543504 4018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "852563c4bc5e4d8ab144dc7a537c963b"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "b64e31464dd6f081bc6a5170239787d1"
server_key_iv: "0bd3a18965797bc53648d4c44967500a"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.543639 4018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:14.566372 4018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:14.567166 4018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:14.567394 4018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:14.568090 4018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:14.569252 4018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:14.569326 4018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.569402 4018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:14.569454 4018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.579546 4018 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:37373
I20260808 04:46:14.579566 4200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:37373 every 8 connection(s)
I20260808 04:46:14.580724 4018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:46:14.589108 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 4018
I20260808 04:46:14.589252 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:46:14.589604 22538 external_mini_cluster.cc:1428] Setting key 9c641b6c67fcdaab96407b5a09bdadfb
I20260808 04:46:14.590557 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:14.582813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:41361 (local address 127.22.2.190:40261)
0808 04:46:14.582983 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:14.582988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:14.584187 (+ 1199us) server_negotiation.cc:407] Connection header received
0808 04:46:14.585577 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:14.585580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:14.585630 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:14.585713 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:14.586684 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.587199 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.588344 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.588607 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.588686 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:14.589906 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:14.589910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:14.589913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:46:14.589926 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:14.589949 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:46:14.589959 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:14.590040 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:14.590341 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 04:46:14.590403 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 04:46:14.591722 4203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:14.583269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40261 (local address 127.22.2.130:41361)
0808 04:46:14.583996 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:14.584037 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:14.585366 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:14.585917 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:14.585924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:14.586016 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:14.586553 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:14.586566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.587314 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.587317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:14.588236 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:14.588246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.588751 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.588754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:14.588895 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:14.589556 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:46:14.589558 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:14.589608 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:46:14.589680 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:14.590129 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:14.590141 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:14.590150 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 04:46:14.590322 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 04:46:14.590649 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":126,"threads_started":1}
I20260808 04:46:14.592170 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40261
--builtin_ntp_servers=127.22.2.148:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 04:46:14.592998 4201 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40261
I20260808 04:46:14.593286 4201 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:14.593829 4201 heartbeater.cc:511] Master 127.22.2.190:40261 requested a full tablet report, sending...
I20260808 04:46:14.594924 3738 ts_manager.cc:194] Registered new tserver with Master: 852563c4bc5e4d8ab144dc7a537c963b (127.22.2.130:37373)
I20260808 04:46:14.595494 3738 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.130:41361
W20260808 04:46:14.709277 4205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:14.709585 4205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:14.709674 4205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:14.713737 4205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:14.713810 4205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:14.713899 4205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:46:14.718513 4205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:14.719578 4205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:14.721067 4205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:14.731015 4217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:14.731056 4214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:14.731047 4213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:14.731210 4205 server_base.cc:1034] running on GCE node
I20260808 04:46:14.731736 4205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:14.732415 4205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:14.733664 4205 hybrid_clock.cc:648] HybridClock initialized: now 1786164374733629 us; error 44 us; skew 500 ppm
I20260808 04:46:14.735760 4205 webserver.cc:460] Webserver started at http://127.22.2.131:44025/ using document root <none> and password file <none>
I20260808 04:46:14.736408 4205 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:14.736508 4205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:14.736775 4205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:14.738969 4205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "4cdea2a429ff4d199cf9cd68171aeeeb"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "4bc1dc8b660a7b8541b2a32df21bd522"
server_key_iv: "c0a2acdb6241237dc7b9ed89d2e5addc"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.739730 4205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "4cdea2a429ff4d199cf9cd68171aeeeb"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "4bc1dc8b660a7b8541b2a32df21bd522"
server_key_iv: "c0a2acdb6241237dc7b9ed89d2e5addc"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.744099 4205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260808 04:46:14.746980 4227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.748437 4205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:46:14.748616 4205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "4cdea2a429ff4d199cf9cd68171aeeeb"
format_stamp: "Formatted at 2026-08-08 04:46:14 on dist-test-slave-04bb"
server_key: "4bc1dc8b660a7b8541b2a32df21bd522"
server_key_iv: "c0a2acdb6241237dc7b9ed89d2e5addc"
server_key_version: "encryptionkey@0"
I20260808 04:46:14.748756 4205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:14.773916 4205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:14.774710 4205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:14.774940 4205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:14.775616 4205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:14.776681 4205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:14.776757 4205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.776836 4205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:14.776881 4205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:14.786509 4205 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:32867
I20260808 04:46:14.786532 4392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:32867 every 8 connection(s)
I20260808 04:46:14.787484 4205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:46:14.790781 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 4205
I20260808 04:46:14.790916 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:46:14.791249 22538 external_mini_cluster.cc:1428] Setting key 61ebf6a14c2051af6b988907d831ff08
I20260808 04:46:14.797784 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:14.789522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:41637 (local address 127.22.2.190:40261)
0808 04:46:14.789674 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:14.789678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:14.790963 (+ 1285us) server_negotiation.cc:407] Connection header received
0808 04:46:14.792680 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:14.792687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:14.792755 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:14.792845 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:14.793963 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.794499 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.795660 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.795904 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.796005 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:14.797268 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:14.797272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:14.797274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:46:14.797295 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:14.797318 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:46:14.797332 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:14.797414 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:14.797596 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 04:46:14.797657 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 04:46:14.798636 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:14.789872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40261 (local address 127.22.2.131:41637)
0808 04:46:14.790760 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:14.790822 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:14.792378 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:14.793061 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:14.793069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:14.793157 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:14.793825 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:14.793843 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.794614 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.794617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:14.795517 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:14.795526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:14.796058 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:14.796060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:14.796203 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:14.796970 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:46:14.796971 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:14.797014 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:46:14.797083 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:14.797422 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:14.797430 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:14.797435 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:46:14.797540 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 04:46:14.797779 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":120,"threads_started":1}
I20260808 04:46:14.799881 4393 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40261
I20260808 04:46:14.800117 4393 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:14.800694 4393 heartbeater.cc:511] Master 127.22.2.190:40261 requested a full tablet report, sending...
I20260808 04:46:14.801764 3738 ts_manager.cc:194] Registered new tserver with Master: 4cdea2a429ff4d199cf9cd68171aeeeb (127.22.2.131:32867)
I20260808 04:46:14.802385 3738 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.131:41637
I20260808 04:46:14.805639 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 04:46:14.808825 3798 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:14.807749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49860 (local address 127.22.2.190:40261)
0808 04:46:14.807910 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:14.807914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:14.808184 (+ 270us) server_negotiation.cc:407] Connection header received
0808 04:46:14.808301 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:14.808305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:14.808350 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:14.808420 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:14.808710 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49860: BlockingRecv error: recv got EOF from 127.0.0.1:49860 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 04:46:14.809280 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 3826
I20260808 04:46:14.821015 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 4018
I20260808 04:46:14.848165 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 4205
I20260808 04:46:14.858628 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 3701
2026-08-08T04:46:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (968 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T04:46:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:46:14Z Disabled control of system clock
I20260808 04:46:14.895489 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33355
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:33355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 04:46:15.009363 4408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:15.009660 4408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:15.009697 4408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:15.013785 4408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:46:15.013852 4408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:15.013867 4408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:15.013885 4408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:46:15.013900 4408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:46:15.018630 4408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:33355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:15.019678 4408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:15.021157 4408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:15.029987 4422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:15.029974 4416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:15.029978 4417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:15.030369 4408 server_base.cc:1034] running on GCE node
I20260808 04:46:15.030972 4408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:15.032024 4408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:15.033183 4408 hybrid_clock.cc:648] HybridClock initialized: now 1786164375033153 us; error 44 us; skew 500 ppm
I20260808 04:46:15.035099 4408 webserver.cc:460] Webserver started at http://127.22.2.190:45323/ using document root <none> and password file <none>
I20260808 04:46:15.035647 4408 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:15.035708 4408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:15.035874 4408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:15.037954 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "e1a48667e60f4226bbeba8d7b397c92e"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "d3cc2d83d6975be6058b5b1c3a66dad8"
server_key_iv: "0bbe230c5d5b756fab2fb04185a25d07"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.038487 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "e1a48667e60f4226bbeba8d7b397c92e"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "d3cc2d83d6975be6058b5b1c3a66dad8"
server_key_iv: "0bbe230c5d5b756fab2fb04185a25d07"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.042256 4408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 04:46:15.045081 4433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.046362 4408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:46:15.046470 4408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "e1a48667e60f4226bbeba8d7b397c92e"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "d3cc2d83d6975be6058b5b1c3a66dad8"
server_key_iv: "0bbe230c5d5b756fab2fb04185a25d07"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.046557 4408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:15.051131 4408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:15.051698 4408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:15.051839 4408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:15.058902 4408 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:33355
I20260808 04:46:15.058926 4500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:33355 every 8 connection(s)
I20260808 04:46:15.059881 4408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:46:15.061355 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 4408
I20260808 04:46:15.061491 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:46:15.061772 22538 external_mini_cluster.cc:1428] Setting key f9e607a9fcbd71cc2fa17136104cf0f2
I20260808 04:46:15.063928 4502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:46:15.068838 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:15.063266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.22.2.190:33355)
0808 04:46:15.064029 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:15.064056 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:15.064103 (+ 47us) server_negotiation.cc:407] Connection header received
0808 04:46:15.065095 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:15.065121 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:15.065589 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:15.066002 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:15.066902 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:15.066909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:15.066925 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:46:15.066938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:15.067009 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:46:15.067034 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:15.067220 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:15.067309 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 04:46:15.067598 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":125,"threads_started":1}
I20260808 04:46:15.072986 4502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e: Bootstrap starting.
I20260808 04:46:15.075510 4502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e: Neither blocks nor log segments found. Creating new log.
I20260808 04:46:15.076447 4502 log.cc:824] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e: Log is configured to *not* fsync() on all Append() calls
I20260808 04:46:15.078504 4502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e: No bootstrap required, opened a new log
I20260808 04:46:15.082455 4502 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a48667e60f4226bbeba8d7b397c92e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33355 } }
I20260808 04:46:15.082702 4502 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:46:15.082780 4502 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1a48667e60f4226bbeba8d7b397c92e, State: Initialized, Role: FOLLOWER
I20260808 04:46:15.083328 4502 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a48667e60f4226bbeba8d7b397c92e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33355 } }
I20260808 04:46:15.083541 4502 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:46:15.083638 4502 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:46:15.083784 4502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:46:15.084875 4502 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a48667e60f4226bbeba8d7b397c92e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33355 } }
I20260808 04:46:15.085261 4502 leader_election.cc:302] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1a48667e60f4226bbeba8d7b397c92e; no voters:
I20260808 04:46:15.085602 4502 leader_election.cc:288] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:46:15.085738 4510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:46:15.086032 4510 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [term 1 LEADER]: Becoming Leader. State: Replica: e1a48667e60f4226bbeba8d7b397c92e, State: Running, Role: LEADER
I20260808 04:46:15.086354 4510 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a48667e60f4226bbeba8d7b397c92e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33355 } }
I20260808 04:46:15.087033 4502 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:46:15.088197 4514 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1a48667e60f4226bbeba8d7b397c92e. Latest consensus state: current_term: 1 leader_uuid: "e1a48667e60f4226bbeba8d7b397c92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a48667e60f4226bbeba8d7b397c92e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33355 } } }
I20260808 04:46:15.088351 4514 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [sys.catalog]: This master's current role is: LEADER
I20260808 04:46:15.088191 4512 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1a48667e60f4226bbeba8d7b397c92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a48667e60f4226bbeba8d7b397c92e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33355 } } }
I20260808 04:46:15.088672 4512 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e [sys.catalog]: This master's current role is: LEADER
I20260808 04:46:15.088788 4525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:46:15.092419 4525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:46:15.098457 4525 catalog_manager.cc:1347] Generated new cluster ID: b55eba5076934a54a6b9b884c7413668
I20260808 04:46:15.098527 4525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:46:15.118717 4525 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:46:15.119058 4525 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:46:15.127943 4525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1a48667e60f4226bbeba8d7b397c92e: Generated new TSK 0
I20260808 04:46:15.128824 4525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:46:15.137622 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33355
--builtin_ntp_servers=127.22.2.148:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 04:46:15.259933 4538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:15.260203 4538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:15.260245 4538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:15.264114 4538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:15.264212 4538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:15.264305 4538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:46:15.269024 4538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:15.269981 4538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:15.271320 4538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:15.280917 4549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:15.280962 4546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:15.280953 4543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:15.281096 4538 server_base.cc:1034] running on GCE node
I20260808 04:46:15.281620 4538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:15.282280 4538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:15.283470 4538 hybrid_clock.cc:648] HybridClock initialized: now 1786164375283444 us; error 41 us; skew 500 ppm
I20260808 04:46:15.285384 4538 webserver.cc:460] Webserver started at http://127.22.2.129:45215/ using document root <none> and password file <none>
I20260808 04:46:15.285991 4538 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:15.286088 4538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:15.286336 4538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:15.288749 4538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "8c0a26de041b43d8bce506eb1038cb9f"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "fbd09e040ee9214bfae234249c7c0cba"
server_key_iv: "34bb1092f748eb8ba1f918ea99b5279e"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.289369 4538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "8c0a26de041b43d8bce506eb1038cb9f"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "fbd09e040ee9214bfae234249c7c0cba"
server_key_iv: "34bb1092f748eb8ba1f918ea99b5279e"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.293128 4538 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:46:15.295753 4557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.297014 4538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:46:15.297161 4538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "8c0a26de041b43d8bce506eb1038cb9f"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "fbd09e040ee9214bfae234249c7c0cba"
server_key_iv: "34bb1092f748eb8ba1f918ea99b5279e"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.297277 4538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:15.301874 4538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:15.302476 4538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:15.302667 4538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:15.303205 4538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:15.304113 4538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:15.304186 4538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.304256 4538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:15.304311 4538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.313294 4538 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:41391
I20260808 04:46:15.313341 4710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:41391 every 8 connection(s)
I20260808 04:46:15.314185 4538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:46:15.315200 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 4538
I20260808 04:46:15.315294 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:46:15.315536 22538 external_mini_cluster.cc:1428] Setting key d1fab42e24c30b61d0c81e0eb6562690
I20260808 04:46:15.317938 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33355
--builtin_ntp_servers=127.22.2.148:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 04:46:15.321070 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:15.316191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:49419 (local address 127.22.2.190:33355)
0808 04:46:15.316356 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:15.316360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:15.317657 (+ 1297us) server_negotiation.cc:407] Connection header received
0808 04:46:15.319080 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:15.319083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:15.319137 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:15.319253 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:15.320613 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:15.320617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:15.320619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:46:15.320631 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:15.320652 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:46:15.320660 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:15.320739 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:15.320924 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 04:46:15.320962 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 04:46:15.322307 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:15.316723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33355 (local address 127.22.2.129:49419)
0808 04:46:15.317510 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:15.317550 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:15.318870 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:15.319454 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:15.319466 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:15.319618 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:15.320245 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:46:15.320248 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:15.320307 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:46:15.320401 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:15.320755 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:15.320766 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:15.320773 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 04:46:15.320969 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 04:46:15.321319 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":119,"threads_started":1}
I20260808 04:46:15.323606 4711 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33355
I20260808 04:46:15.323875 4711 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:15.324391 4711 heartbeater.cc:511] Master 127.22.2.190:33355 requested a full tablet report, sending...
I20260808 04:46:15.326040 4455 ts_manager.cc:194] Registered new tserver with Master: 8c0a26de041b43d8bce506eb1038cb9f (127.22.2.129:41391)
W20260808 04:46:15.433045 4715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:15.433324 4715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:15.433394 4715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:15.437424 4715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:15.437495 4715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:15.437583 4715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:46:15.442238 4715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:15.443293 4715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:15.444720 4715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:15.454583 4720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:15.454612 4721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:15.454583 4723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:15.454779 4715 server_base.cc:1034] running on GCE node
I20260808 04:46:15.455143 4715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:15.455801 4715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:15.457062 4715 hybrid_clock.cc:648] HybridClock initialized: now 1786164375457048 us; error 28 us; skew 500 ppm
I20260808 04:46:15.458974 4715 webserver.cc:460] Webserver started at http://127.22.2.130:39865/ using document root <none> and password file <none>
I20260808 04:46:15.459570 4715 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:15.459666 4715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:15.459913 4715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:15.462003 4715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "050296252a624d9ea3d790b2bf9f8868"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "cdc52f9bcf819eb5ea3bc8016a63ad50"
server_key_iv: "9cfdd250dad213ad3e3453ed591b989c"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.462584 4715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "050296252a624d9ea3d790b2bf9f8868"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "cdc52f9bcf819eb5ea3bc8016a63ad50"
server_key_iv: "9cfdd250dad213ad3e3453ed591b989c"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.466281 4715 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:46:15.468789 4729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.469949 4715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:46:15.470085 4715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "050296252a624d9ea3d790b2bf9f8868"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "cdc52f9bcf819eb5ea3bc8016a63ad50"
server_key_iv: "9cfdd250dad213ad3e3453ed591b989c"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.470216 4715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:15.474745 4715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:15.475340 4715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:15.475524 4715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:15.476055 4715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:15.477061 4715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:15.477133 4715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.477212 4715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:15.477257 4715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.486019 4715 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:39505
I20260808 04:46:15.486050 4890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:39505 every 8 connection(s)
I20260808 04:46:15.486953 4715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:46:15.493264 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:15.489093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:57055 (local address 127.22.2.190:33355)
0808 04:46:15.489248 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:15.489252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:15.490323 (+ 1071us) server_negotiation.cc:407] Connection header received
0808 04:46:15.491628 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:15.491632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:15.491673 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:15.491752 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:15.492878 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:15.492881 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:15.492883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:46:15.492892 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:15.492907 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:46:15.492914 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:15.492986 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:15.493109 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 04:46:15.493145 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 04:46:15.494067 4894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:15.489540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33355 (local address 127.22.2.130:57055)
0808 04:46:15.490196 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:15.490235 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:15.491418 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:15.491923 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:15.491930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:15.492013 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:15.492641 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:46:15.492643 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:15.492684 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:46:15.492750 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:15.492990 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:15.492997 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:15.493001 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:46:15.493129 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 04:46:15.493321 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
I20260808 04:46:15.495213 4891 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33355
I20260808 04:46:15.495424 4891 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:15.495697 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 4715
I20260808 04:46:15.495821 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:46:15.495901 4891 heartbeater.cc:511] Master 127.22.2.190:33355 requested a full tablet report, sending...
I20260808 04:46:15.496150 22538 external_mini_cluster.cc:1428] Setting key e7ef05b1e5abb49fc011e22b4049877a
I20260808 04:46:15.497051 4455 ts_manager.cc:194] Registered new tserver with Master: 050296252a624d9ea3d790b2bf9f8868 (127.22.2.130:39505)
I20260808 04:46:15.498219 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33355
--builtin_ntp_servers=127.22.2.148:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 04:46:15.614267 4895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:15.614526 4895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:15.614598 4895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:15.618553 4895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:15.618606 4895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:15.618660 4895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:46:15.623225 4895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:15.624295 4895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:15.625694 4895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:15.635419 4900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:15.635428 4901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:15.635704 4895 server_base.cc:1034] running on GCE node
W20260808 04:46:15.635473 4904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:15.636179 4895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:15.636844 4895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:15.638026 4895 hybrid_clock.cc:648] HybridClock initialized: now 1786164375638002 us; error 47 us; skew 500 ppm
I20260808 04:46:15.639866 4895 webserver.cc:460] Webserver started at http://127.22.2.131:43057/ using document root <none> and password file <none>
I20260808 04:46:15.640452 4895 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:15.640565 4895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:15.640818 4895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:15.642804 4895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "959894598730470aa0a65c98f3e6e7e5"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "fed274550f62c75c35e0ff66a9d6e777"
server_key_iv: "618657392dc058f2ee1908bfbdf782f3"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.643386 4895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "959894598730470aa0a65c98f3e6e7e5"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "fed274550f62c75c35e0ff66a9d6e777"
server_key_iv: "618657392dc058f2ee1908bfbdf782f3"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.647018 4895 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260808 04:46:15.649583 4911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.650887 4895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:46:15.651053 4895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "959894598730470aa0a65c98f3e6e7e5"
format_stamp: "Formatted at 2026-08-08 04:46:15 on dist-test-slave-04bb"
server_key: "fed274550f62c75c35e0ff66a9d6e777"
server_key_iv: "618657392dc058f2ee1908bfbdf782f3"
server_key_version: "encryptionkey@0"
I20260808 04:46:15.651162 4895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:15.655658 4895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:15.656250 4895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:15.656441 4895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:15.657017 4895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:15.657909 4895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:15.657958 4895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.657994 4895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:15.658043 4895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:15.666708 4895 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:41975
I20260808 04:46:15.666733 5071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:41975 every 8 connection(s)
I20260808 04:46:15.667610 4895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:46:15.673767 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:15.669574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:60925 (local address 127.22.2.190:33355)
0808 04:46:15.669722 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:15.669725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:15.670733 (+ 1008us) server_negotiation.cc:407] Connection header received
0808 04:46:15.672006 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:15.672009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:15.672049 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:15.672127 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:15.673361 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:15.673365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:15.673366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:46:15.673378 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:15.673397 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:46:15.673406 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:15.673486 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:15.673609 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 04:46:15.673646 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:46:15.674609 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:15.669916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33355 (local address 127.22.2.131:60925)
0808 04:46:15.670575 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:15.670615 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:15.671783 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:15.672293 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:15.672300 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:15.672385 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:15.673091 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:46:15.673092 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:15.673132 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:46:15.673201 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:15.673488 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:15.673494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:15.673499 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:46:15.673627 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 04:46:15.673823 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":120,"threads_started":1}
I20260808 04:46:15.675302 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 4895
I20260808 04:46:15.675416 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:46:15.675714 22538 external_mini_cluster.cc:1428] Setting key d4f85e7f2548ed761fcad54c83fccd5d
I20260808 04:46:15.675889 5072 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33355
I20260808 04:46:15.676124 5072 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:15.676630 5072 heartbeater.cc:511] Master 127.22.2.190:33355 requested a full tablet report, sending...
I20260808 04:46:15.677757 4455 ts_manager.cc:194] Registered new tserver with Master: 959894598730470aa0a65c98f3e6e7e5 (127.22.2.131:41975)
I20260808 04:46:15.690105 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 04:46:15.693126 4508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:46:15.692160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42640 (local address 127.22.2.190:33355)
0808 04:46:15.692304 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:15.692308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:15.692494 (+ 186us) server_negotiation.cc:407] Connection header received
0808 04:46:15.692649 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:15.692652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:15.692687 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:15.692783 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:15.692976 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42640: BlockingRecv error: recv got EOF from 127.0.0.1:42640 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 04:46:15.693650 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 4538
I20260808 04:46:15.711318 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 4715
I20260808 04:46:15.723131 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 4895
I20260808 04:46:15.733891 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 4408
2026-08-08T04:46:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (873 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:46:15 dist-test-slave-04bb krb5kdc[5088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:46:15 dist-test-slave-04bb krb5kdc[5088](info): set up 2 sockets
Aug 08 04:46:15 dist-test-slave-04bb krb5kdc[5088](info): commencing operation
krb5kdc: starting...
W20260808 04:46:17.001995 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:46:17 dist-test-slave-04bb krb5kdc[5088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:46:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:46:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:46:17.131585 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:43629
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:43629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:46:17.244915 5111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:17.245193 5111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:17.245232 5111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:17.249197 5111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:46:17.249244 5111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:17.249262 5111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:17.249279 5111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:46:17.249325 5111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:46:17.254426 5111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:43629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:43629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:17.255419 5111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:17.256434 5111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:17.265197 5118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:17.265257 5121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:17.265257 5116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:17.265439 5111 server_base.cc:1034] running on GCE node
I20260808 04:46:17.266062 5111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:17.267128 5111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:17.268326 5111 hybrid_clock.cc:648] HybridClock initialized: now 1786164377268308 us; error 37 us; skew 500 ppm
Aug 08 04:46:17 dist-test-slave-04bb krb5kdc[5088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164377, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:46:17.271190 5111 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:46:17.272104 5111 webserver.cc:460] Webserver started at http://127.22.2.190:43757/ using document root <none> and password file <none>
I20260808 04:46:17.272730 5111 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:17.272847 5111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:17.273079 5111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:17.275081 5111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "d718bd486d5649cdb04e7b03b341dbd3"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "73fc6aba717d65e05923aa5bffe987ef"
server_key_iv: "7c4f4d8d128e933235a3ce9a5050fde6"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.275651 5111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "d718bd486d5649cdb04e7b03b341dbd3"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "73fc6aba717d65e05923aa5bffe987ef"
server_key_iv: "7c4f4d8d128e933235a3ce9a5050fde6"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.279366 5111 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:46:17.281854 5135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:17.282997 5111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:46:17.283138 5111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "d718bd486d5649cdb04e7b03b341dbd3"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "73fc6aba717d65e05923aa5bffe987ef"
server_key_iv: "7c4f4d8d128e933235a3ce9a5050fde6"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.283248 5111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:17.312624 5111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:17.316121 5111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:17.316335 5111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:17.323843 5111 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:43629
I20260808 04:46:17.323901 5220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:43629 every 8 connection(s)
I20260808 04:46:17.324892 5111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:46:17.327759 5221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:46:17.328449 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 5111
I20260808 04:46:17.328604 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:46:17.328900 22538 external_mini_cluster.cc:1428] Setting key 59d640905b574fca73098071d5c3adc5
I20260808 04:46:17.334702 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3: Bootstrap starting.
I20260808 04:46:17.337265 5221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3: Neither blocks nor log segments found. Creating new log.
I20260808 04:46:17.338045 5221 log.cc:824] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:46:17 dist-test-slave-04bb krb5kdc[5088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:46:17.340251 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3: No bootstrap required, opened a new log
I20260808 04:46:17.343684 5221 raft_consensus.cc:348] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d718bd486d5649cdb04e7b03b341dbd3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43629 } }
I20260808 04:46:17.343947 5221 raft_consensus.cc:374] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:46:17.344012 5221 raft_consensus.cc:729] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d718bd486d5649cdb04e7b03b341dbd3, State: Initialized, Role: FOLLOWER
I20260808 04:46:17.344468 5221 consensus_queue.cc:260] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d718bd486d5649cdb04e7b03b341dbd3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43629 } }
I20260808 04:46:17.344647 5221 raft_consensus.cc:388] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:46:17.344733 5221 raft_consensus.cc:482] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:46:17.344839 5221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:46:17.344861 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:17.330370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55622 (local address 127.22.2.190:43629)
0808 04:46:17.330819 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:17.330836 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:17.330872 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:46:17.331511 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:17.331528 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:17.331848 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:17.332226 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:17.333228 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.334727 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.335590 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.336339 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.336501 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:17.339220 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:17.339233 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:17.339247 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:46:17.339273 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:17.341360 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:17.341922 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:17.341930 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:17.341937 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:17.342085 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:17.342433 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:17.342439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:17.342441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:17.342873 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:17.343145 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:17.343489 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 04:46:17.343814 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":137,"threads_started":1}
I20260808 04:46:17.345736 5221 raft_consensus.cc:504] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d718bd486d5649cdb04e7b03b341dbd3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43629 } }
I20260808 04:46:17.346086 5221 leader_election.cc:302] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d718bd486d5649cdb04e7b03b341dbd3; no voters:
I20260808 04:46:17.346405 5221 leader_election.cc:288] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:46:17.346549 5230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:46:17.346805 5230 raft_consensus.cc:686] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [term 1 LEADER]: Becoming Leader. State: Replica: d718bd486d5649cdb04e7b03b341dbd3, State: Running, Role: LEADER
I20260808 04:46:17.347139 5230 consensus_queue.cc:237] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d718bd486d5649cdb04e7b03b341dbd3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43629 } }
I20260808 04:46:17.347740 5221 sys_catalog.cc:564] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:46:17.348436 5230 sys_catalog.cc:455] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d718bd486d5649cdb04e7b03b341dbd3. Latest consensus state: current_term: 1 leader_uuid: "d718bd486d5649cdb04e7b03b341dbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d718bd486d5649cdb04e7b03b341dbd3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43629 } } }
I20260808 04:46:17.348608 5230 sys_catalog.cc:458] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [sys.catalog]: This master's current role is: LEADER
I20260808 04:46:17.349005 5231 sys_catalog.cc:455] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d718bd486d5649cdb04e7b03b341dbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d718bd486d5649cdb04e7b03b341dbd3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43629 } } }
I20260808 04:46:17.349123 5231 sys_catalog.cc:458] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3 [sys.catalog]: This master's current role is: LEADER
W20260808 04:46:17.353025 5257 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:46:17.353106 5257 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:46:17.353233 5248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:46:17.354000 5248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:46:17.359649 5248 catalog_manager.cc:1347] Generated new cluster ID: 1021912572404b6192525f51de725ab9
I20260808 04:46:17.359725 5248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:46:17.393687 5248 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:46:17.395027 5248 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:46:17.404541 5248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3: Generated new TSK 0
I20260808 04:46:17.405329 5248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:46:17.451476 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43629
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:46:17.570724 5266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:17.570986 5266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:17.571033 5266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:17.574877 5266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:17.574932 5266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:17.574991 5266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:46:17.580061 5266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:17.581041 5266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:17.582003 5266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:17.591956 5279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:17.591943 5275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:17.591928 5273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:17.592237 5266 server_base.cc:1034] running on GCE node
I20260808 04:46:17.592727 5266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:17.593367 5266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:17.594530 5266 hybrid_clock.cc:648] HybridClock initialized: now 1786164377594520 us; error 30 us; skew 500 ppm
Aug 08 04:46:17 dist-test-slave-04bb krb5kdc[5088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164377, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:46:17.597446 5266 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:46:17.598369 5266 webserver.cc:460] Webserver started at http://127.22.2.129:39401/ using document root <none> and password file <none>
I20260808 04:46:17.598933 5266 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:17.598995 5266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:17.599249 5266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:17.601325 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "5a3f45bbe328493682e6f9bee70382e2"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "7c0c4b631952bbfce456a3d2e2a0807e"
server_key_iv: "c782515afb3a19a68827a1c370b835af"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.601884 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "5a3f45bbe328493682e6f9bee70382e2"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "7c0c4b631952bbfce456a3d2e2a0807e"
server_key_iv: "c782515afb3a19a68827a1c370b835af"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.605532 5266 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:46:17.607913 5293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:17.609176 5266 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:46:17.609323 5266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "5a3f45bbe328493682e6f9bee70382e2"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "7c0c4b631952bbfce456a3d2e2a0807e"
server_key_iv: "c782515afb3a19a68827a1c370b835af"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.609433 5266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:17.626080 5266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:17.629397 5266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:17.629608 5266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:17.630231 5266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:17.631081 5266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:17.631155 5266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:17.631227 5266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:17.631270 5266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:17.640760 5266 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:44253
I20260808 04:46:17.640817 5467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:44253 every 8 connection(s)
I20260808 04:46:17.641659 5266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:46:17.648901 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 5266
I20260808 04:46:17.649057 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:46:17.649437 22538 external_mini_cluster.cc:1428] Setting key 56266149337891d6ce7c89f8c88aaa54
Aug 08 04:46:17 dist-test-slave-04bb krb5kdc[5088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164377, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:46:17.656693 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:17.643586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:53493 (local address 127.22.2.190:43629)
0808 04:46:17.643741 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:17.643745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:17.644594 (+ 849us) server_negotiation.cc:407] Connection header received
0808 04:46:17.646062 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:17.646065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:17.646114 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:17.646190 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:17.647360 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.648029 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.649568 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.649870 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.650000 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:17.652831 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:17.652837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:17.652843 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:46:17.652878 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:17.654359 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:17.654973 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:17.654977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:17.654978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:17.655069 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:17.655456 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:17.655460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:17.655462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:17.655673 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:17.655799 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:17.656331 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 04:46:17.656470 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 04:46:17.657502 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:17.643925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43629 (local address 127.22.2.129:53493)
0808 04:46:17.644419 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:17.644464 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:17.645856 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:17.646335 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:17.646342 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:17.646714 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:17.647217 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:17.647231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.648154 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.648157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:17.649419 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:17.649431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.650062 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.650066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:17.650260 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:17.650913 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:17.650933 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:17.652650 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:17.654487 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:46:17.654495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:46:17.654512 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:46:17.654838 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:46:17.655191 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:46:17.655194 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:46:17.655196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:46:17.655339 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:46:17.655797 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:17.655805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:17.656055 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 04:46:17.656269 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 04:46:17.656624 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":106,"threads_started":1}
I20260808 04:46:17.658963 5469 heartbeater.cc:347] Connected to a master server at 127.22.2.190:43629
I20260808 04:46:17.659248 5469 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:17.659754 5469 heartbeater.cc:511] Master 127.22.2.190:43629 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:46:17.661484 5167 ts_manager.cc:194] Registered new tserver with Master: 5a3f45bbe328493682e6f9bee70382e2 (127.22.2.129:44253)
I20260808 04:46:17.662679 5167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:53493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:46:17.679782 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43629
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:46:17.792990 5477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:17.793246 5477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:17.793290 5477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:17.797262 5477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:17.797318 5477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:17.797375 5477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:46:17.802546 5477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:17.803547 5477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:17.804545 5477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:17.814425 5487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:17.814457 5495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:17.814581 5477 server_base.cc:1034] running on GCE node
W20260808 04:46:17.814461 5488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:17.815140 5477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:17.815833 5477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:17.817014 5477 hybrid_clock.cc:648] HybridClock initialized: now 1786164377817001 us; error 28 us; skew 500 ppm
Aug 08 04:46:17 dist-test-slave-04bb krb5kdc[5088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164377, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:46:17.820279 5477 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:46:17.821211 5477 webserver.cc:460] Webserver started at http://127.22.2.130:37741/ using document root <none> and password file <none>
I20260808 04:46:17.821756 5477 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:17.821822 5477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:17.822003 5477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:17.824088 5477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "65549a4f1d274bada333aca776eeeba7"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "d4b20309e725a23d805ceac15f33c5d0"
server_key_iv: "535d7dc95b7ce08e82c022018d89837e"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.824745 5477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "65549a4f1d274bada333aca776eeeba7"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "d4b20309e725a23d805ceac15f33c5d0"
server_key_iv: "535d7dc95b7ce08e82c022018d89837e"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.828730 5477 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:46:17.831200 5507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:17.832464 5477 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:46:17.832842 5477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "65549a4f1d274bada333aca776eeeba7"
format_stamp: "Formatted at 2026-08-08 04:46:17 on dist-test-slave-04bb"
server_key: "d4b20309e725a23d805ceac15f33c5d0"
server_key_iv: "535d7dc95b7ce08e82c022018d89837e"
server_key_version: "encryptionkey@0"
I20260808 04:46:17.832976 5477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:17.844429 5477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:17.847378 5477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:17.847635 5477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:17.848310 5477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:17.849431 5477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:17.849514 5477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:17.849601 5477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:17.849649 5477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:17.859434 5477 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:39657
I20260808 04:46:17.859467 5668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:39657 every 8 connection(s)
I20260808 04:46:17.860453 5477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:46:17.866464 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 5477
I20260808 04:46:17.866596 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:46:17.866936 22538 external_mini_cluster.cc:1428] Setting key fe982923cd0f8817aa76c0eb7519effa
Aug 08 04:46:17 dist-test-slave-04bb krb5kdc[5088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164377, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:46:17.876768 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:17.862383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53347 (local address 127.22.2.190:43629)
0808 04:46:17.862524 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:17.862528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:17.863418 (+ 890us) server_negotiation.cc:407] Connection header received
0808 04:46:17.864875 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:17.864878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:17.864926 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:17.865002 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:17.866185 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.866780 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.868088 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.868418 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.868569 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:17.871656 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:17.871663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:17.871667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:46:17.871699 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:17.873682 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:17.874418 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:17.874424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:17.874428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:17.874536 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:17.874977 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:17.874982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:17.874985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:17.875280 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:17.875413 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:17.876392 (+ 979us) server_negotiation.cc:299] Negotiation successful
0808 04:46:17.876563 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 04:46:17.877511 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:17.862736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43629 (local address 127.22.2.130:53347)
0808 04:46:17.863247 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:17.863292 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:17.864646 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:17.865207 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:17.865214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:17.865559 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:17.866048 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:17.866061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.866915 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.866919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:17.867965 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:17.867975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:17.868614 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:17.868618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:17.868842 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:17.869463 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:17.869490 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:17.871479 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:17.873828 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:46:17.873835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:46:17.873850 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:46:17.874253 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:46:17.874657 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:46:17.874660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:46:17.874663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:46:17.874861 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:46:17.875454 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:17.875499 (+ 45us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:17.876080 (+ 581us) client_negotiation.cc:769] Sending connection context
0808 04:46:17.876283 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 04:46:17.876605 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:46:17.878926 5672 heartbeater.cc:347] Connected to a master server at 127.22.2.190:43629
I20260808 04:46:17.879154 5672 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:17.879722 5672 heartbeater.cc:511] Master 127.22.2.190:43629 requested a full tablet report, sending...
I20260808 04:46:17.880924 5167 ts_manager.cc:194] Registered new tserver with Master: 65549a4f1d274bada333aca776eeeba7 (127.22.2.130:39657)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:46:17.881641 5167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:53347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:46:17.896982 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43629
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:46:18.012152 5679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:46:18.012396 5679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:46:18.012437 5679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:46:18.016585 5679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:46:18.016646 5679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:46:18.016703 5679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:46:18.021903 5679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:46:18.022938 5679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:46:18.023939 5679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:46:18.033923 5691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:46:18.033926 5686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:18.034106 5679 server_base.cc:1034] running on GCE node
W20260808 04:46:18.033923 5687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:46:18.034746 5679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:46:18.035398 5679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:46:18.036613 5679 hybrid_clock.cc:648] HybridClock initialized: now 1786164378036597 us; error 28 us; skew 500 ppm
Aug 08 04:46:18 dist-test-slave-04bb krb5kdc[5088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164378, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:46:18.039490 5679 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:46:18.040441 5679 webserver.cc:460] Webserver started at http://127.22.2.131:39497/ using document root <none> and password file <none>
I20260808 04:46:18.041054 5679 fs_manager.cc:359] Metadata directory not provided
I20260808 04:46:18.041147 5679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:46:18.041395 5679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:46:18.043426 5679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "ce090c61274a4e50a4437f1269219a56"
format_stamp: "Formatted at 2026-08-08 04:46:18 on dist-test-slave-04bb"
server_key: "d297158ff75b98c7d06baeb1f3da5204"
server_key_iv: "06f2ded12b5ca8b03c11b5a12c879430"
server_key_version: "encryptionkey@0"
I20260808 04:46:18.044003 5679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "ce090c61274a4e50a4437f1269219a56"
format_stamp: "Formatted at 2026-08-08 04:46:18 on dist-test-slave-04bb"
server_key: "d297158ff75b98c7d06baeb1f3da5204"
server_key_iv: "06f2ded12b5ca8b03c11b5a12c879430"
server_key_version: "encryptionkey@0"
I20260808 04:46:18.047725 5679 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:46:18.050397 5700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:18.051617 5679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 04:46:18.051754 5679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "ce090c61274a4e50a4437f1269219a56"
format_stamp: "Formatted at 2026-08-08 04:46:18 on dist-test-slave-04bb"
server_key: "d297158ff75b98c7d06baeb1f3da5204"
server_key_iv: "06f2ded12b5ca8b03c11b5a12c879430"
server_key_version: "encryptionkey@0"
I20260808 04:46:18.051869 5679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:46:18.064924 5679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:46:18.068015 5679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:46:18.068216 5679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:46:18.068874 5679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:46:18.069793 5679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:46:18.069867 5679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:18.069938 5679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:46:18.069989 5679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:46:18.079092 5679 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:46067
I20260808 04:46:18.079115 5858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:46067 every 8 connection(s)
I20260808 04:46:18.079998 5679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:46:18.083911 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 5679
I20260808 04:46:18.084004 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:46:18.084309 22538 external_mini_cluster.cc:1428] Setting key f8bd3fa5dd71b2edfa41849bd9f0782e
Aug 08 04:46:18 dist-test-slave-04bb krb5kdc[5088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164378, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:46:18.096660 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.082065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:57373 (local address 127.22.2.190:43629)
0808 04:46:18.082242 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:18.082247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:18.083059 (+ 812us) server_negotiation.cc:407] Connection header received
0808 04:46:18.084705 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:18.084708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:18.084759 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:18.084837 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:18.086475 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.087148 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.088486 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.088783 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.088873 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.091502 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:18.091505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:18.091508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:46:18.091531 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:18.093711 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:18.094455 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:18.094461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:18.094465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:18.094570 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:18.095075 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:18.095081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:18.095082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:18.095368 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:18.095485 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:18.096253 (+ 768us) server_negotiation.cc:299] Negotiation successful
0808 04:46:18.096416 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 04:46:18.097658 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.082441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43629 (local address 127.22.2.131:57373)
0808 04:46:18.082905 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:18.082947 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:18.084388 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:18.085081 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:18.085091 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:18.085590 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:46:18.086300 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.086320 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.087309 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.087313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.088369 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.088379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.088938 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.088941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.089093 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.089723 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:46:18.089741 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:46:18.091324 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:46:18.093847 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:46:18.093854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:46:18.093894 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:46:18.094332 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:46:18.094708 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:46:18.094711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:46:18.094713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:46:18.094943 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:46:18.095484 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:46:18.095490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:46:18.095880 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 04:46:18.096149 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 04:46:18.096515 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":117,"threads_started":1}
I20260808 04:46:18.099454 5859 heartbeater.cc:347] Connected to a master server at 127.22.2.190:43629
I20260808 04:46:18.099700 5859 heartbeater.cc:464] Registering TS with master...
I20260808 04:46:18.100183 5859 heartbeater.cc:511] Master 127.22.2.190:43629 requested a full tablet report, sending...
I20260808 04:46:18.101321 5167 ts_manager.cc:194] Registered new tserver with Master: ce090c61274a4e50a4437f1269219a56 (127.22.2.131:46067)
I20260808 04:46:18.101904 5167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:57373
I20260808 04:46:18.110759 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:46:18.121201 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.112809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.22.2.190:43629)
0808 04:46:18.113008 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:18.113013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:18.113268 (+ 255us) server_negotiation.cc:407] Connection header received
0808 04:46:18.113410 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:18.113414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:18.113476 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:18.113572 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:46:18.114483 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.115053 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.116161 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.116445 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.116587 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.117986 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:46:18.117989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:46:18.117992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:46:18.118020 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:46:18.119408 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:18.119851 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:18.119855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:18.119856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:18.119942 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:46:18.120237 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:46:18.120240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:46:18.120242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:46:18.120470 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:46:18.120648 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:18.120896 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 04:46:18.121012 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260808 04:46:18.124325 5167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:46:18.127485 5167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:46:18.144766 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.138773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:44253 (local address 127.0.0.1:41992)
0808 04:46:18.139101 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:18.139113 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:18.139238 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:18.140044 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:18.140047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:18.140079 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:46:18.140236 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.140246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.141627 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.141631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.142938 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.142949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.144115 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.144119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.144241 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.144291 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 04:46:18.144409 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 04:46:18.144579 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":2,"spinlock_wait_cycles":1280,"thread_start_us":127,"threads_started":1}
I20260808 04:46:18.144938 5881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.138452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:46067 (local address 127.0.0.1:40952)
0808 04:46:18.138941 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:18.138957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:18.139070 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:18.139984 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:18.139992 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:18.140020 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:46:18.140260 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.140270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.142105 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.142110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.143451 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.143461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.144418 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.144422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.144513 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.144574 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 04:46:18.144637 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 04:46:18.144723 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":126,"spinlock_wait_cycles":5888,"thread_start_us":125,"threads_started":1}
I20260808 04:46:18.144980 5877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.137343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:39657 (local address 127.0.0.1:49808)
0808 04:46:18.137729 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:18.137790 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:18.137930 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:18.140604 (+ 2674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:18.140607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:18.140625 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:46:18.140732 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.140739 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.142125 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.142128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.143499 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.143539 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.144647 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.144650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.144744 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.144764 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 04:46:18.144814 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 04:46:18.144870 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":204,"thread_start_us":111,"threads_started":1}
I20260808 04:46:18.145516 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.137871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40952 (local address 127.22.2.131:46067)
0808 04:46:18.138392 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:18.138404 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:18.139207 (+ 803us) server_negotiation.cc:407] Connection header received
0808 04:46:18.139319 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:18.139325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:18.139565 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:18.139738 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:46:18.140427 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.141944 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.143597 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.144273 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.144411 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.144597 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:18.144666 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 04:46:18.144758 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":307,"threads_started":1}
I20260808 04:46:18.145556 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.139312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.22.2.130:39657)
0808 04:46:18.139998 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:18.140012 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:18.140037 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:46:18.140117 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:18.140123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:18.140325 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:18.140643 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:46:18.140827 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.141954 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.143651 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.144477 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.145120 (+ 643us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.145250 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:18.145325 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 04:46:18.145408 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":133,"threads_started":1}
I20260808 04:46:18.145617 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.139293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.22.2.129:44253)
0808 04:46:18.139588 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:18.139603 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:18.139626 (+ 23us) server_negotiation.cc:407] Connection header received
0808 04:46:18.139703 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:18.139707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:18.139887 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:18.140410 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:46:18.140631 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.141468 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.143101 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.143964 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.144077 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.144232 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:18.145321 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0808 04:46:18.145443 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":123,"threads_started":1}
I20260808 04:46:18.148778 5367 tablet_service.cc:1467] Processing CreateTablet for tablet 46e88e6f5f204caaa44ca53332bda8a2 (DEFAULT_TABLE table=test-table [id=267ff3c4458a43b6b475d4acda10074c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:46:18.149245 5767 tablet_service.cc:1467] Processing CreateTablet for tablet 46e88e6f5f204caaa44ca53332bda8a2 (DEFAULT_TABLE table=test-table [id=267ff3c4458a43b6b475d4acda10074c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:46:18.149245 5580 tablet_service.cc:1467] Processing CreateTablet for tablet 46e88e6f5f204caaa44ca53332bda8a2 (DEFAULT_TABLE table=test-table [id=267ff3c4458a43b6b475d4acda10074c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:46:18.149976 5367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e88e6f5f204caaa44ca53332bda8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:46:18.150352 5767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e88e6f5f204caaa44ca53332bda8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:46:18.150343 5580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e88e6f5f204caaa44ca53332bda8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:46:18.156492 5885 tablet_bootstrap.cc:492] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2: Bootstrap starting.
I20260808 04:46:18.157743 5886 tablet_bootstrap.cc:492] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Bootstrap starting.
I20260808 04:46:18.158653 5887 tablet_bootstrap.cc:492] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: Bootstrap starting.
I20260808 04:46:18.159348 5885 tablet_bootstrap.cc:654] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2: Neither blocks nor log segments found. Creating new log.
I20260808 04:46:18.159956 5886 tablet_bootstrap.cc:654] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Neither blocks nor log segments found. Creating new log.
I20260808 04:46:18.160326 5885 log.cc:824] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2: Log is configured to *not* fsync() on all Append() calls
I20260808 04:46:18.161267 5886 log.cc:824] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Log is configured to *not* fsync() on all Append() calls
I20260808 04:46:18.161281 5887 tablet_bootstrap.cc:654] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: Neither blocks nor log segments found. Creating new log.
I20260808 04:46:18.162133 5887 log.cc:824] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: Log is configured to *not* fsync() on all Append() calls
I20260808 04:46:18.162647 5885 tablet_bootstrap.cc:492] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2: No bootstrap required, opened a new log
I20260808 04:46:18.162884 5885 ts_tablet_manager.cc:1397] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260808 04:46:18.163152 5886 tablet_bootstrap.cc:492] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: No bootstrap required, opened a new log
I20260808 04:46:18.163354 5886 ts_tablet_manager.cc:1397] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:46:18.164391 5887 tablet_bootstrap.cc:492] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: No bootstrap required, opened a new log
I20260808 04:46:18.164649 5887 ts_tablet_manager.cc:1397] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:46:18.166424 5886 raft_consensus.cc:348] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.166729 5886 raft_consensus.cc:374] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:46:18.166796 5886 raft_consensus.cc:729] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce090c61274a4e50a4437f1269219a56, State: Initialized, Role: FOLLOWER
I20260808 04:46:18.167253 5886 consensus_queue.cc:260] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.167302 5885 raft_consensus.cc:348] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.167591 5885 raft_consensus.cc:374] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:46:18.167660 5885 raft_consensus.cc:729] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3f45bbe328493682e6f9bee70382e2, State: Initialized, Role: FOLLOWER
I20260808 04:46:18.167896 5859 heartbeater.cc:503] Master 127.22.2.190:43629 was elected leader, sending a full tablet report...
I20260808 04:46:18.168252 5886 ts_tablet_manager.cc:1428] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 04:46:18.168179 5885 consensus_queue.cc:260] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.169224 5885 ts_tablet_manager.cc:1428] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260808 04:46:18.169279 5469 heartbeater.cc:503] Master 127.22.2.190:43629 was elected leader, sending a full tablet report...
I20260808 04:46:18.169337 5887 raft_consensus.cc:348] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.169656 5887 raft_consensus.cc:374] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:46:18.169726 5887 raft_consensus.cc:729] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65549a4f1d274bada333aca776eeeba7, State: Initialized, Role: FOLLOWER
I20260808 04:46:18.170181 5887 consensus_queue.cc:260] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.170871 5672 heartbeater.cc:503] Master 127.22.2.190:43629 was elected leader, sending a full tablet report...
I20260808 04:46:18.171206 5887 ts_tablet_manager.cc:1428] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260808 04:46:18.288733 5899 raft_consensus.cc:482] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:46:18.288957 5899 raft_consensus.cc:504] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.290217 5899 leader_election.cc:288] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a3f45bbe328493682e6f9bee70382e2 (127.22.2.129:44253), ce090c61274a4e50a4437f1269219a56 (127.22.2.131:46067)
I20260808 04:46:18.294082 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.290750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:32945 (local address 127.22.2.129:44253)
0808 04:46:18.290898 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:18.290903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:18.290922 (+ 19us) server_negotiation.cc:407] Connection header received
0808 04:46:18.290984 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:18.290991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:18.291054 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:18.291157 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:46:18.291443 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.291954 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.293136 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.293522 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.293652 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.293710 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:18.293893 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 04:46:18.293956 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 04:46:18.294082 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.290527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:44253 (local address 127.22.2.130:32945)
0808 04:46:18.290676 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:18.290694 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:18.290814 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:18.291174 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:18.291178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:18.291196 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:46:18.291318 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.291324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.292076 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.292079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.293009 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.293019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.293646 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.293650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.293759 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.293777 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 04:46:18.293877 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 04:46:18.293948 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 04:46:18.294443 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.290634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:39887 (local address 127.22.2.131:46067)
0808 04:46:18.290783 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:18.290788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:18.291137 (+ 349us) server_negotiation.cc:407] Connection header received
0808 04:46:18.291282 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:18.291285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:18.291335 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:18.291423 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:46:18.291695 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.292189 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.293371 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.293809 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.293964 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.294036 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:18.294216 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 04:46:18.294296 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 04:46:18.294822 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.290527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:46067 (local address 127.22.2.130:39887)
0808 04:46:18.291047 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:18.291057 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:46:18.291162 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:46:18.291434 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:46:18.291437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:46:18.291453 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:46:18.291577 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.291582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.292306 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.292310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.293247 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:46:18.293257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.293954 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.293958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:46:18.294059 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.294081 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 04:46:18.294582 (+ 501us) client_negotiation.cc:240] Negotiation successful
0808 04:46:18.294661 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":148,"thread_start_us":167,"threads_started":1}
I20260808 04:46:18.294955 5403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e88e6f5f204caaa44ca53332bda8a2" candidate_uuid: "65549a4f1d274bada333aca776eeeba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f45bbe328493682e6f9bee70382e2" is_pre_election: true
I20260808 04:46:18.295343 5403 raft_consensus.cc:2454] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65549a4f1d274bada333aca776eeeba7 in term 0.
I20260808 04:46:18.295624 5796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e88e6f5f204caaa44ca53332bda8a2" candidate_uuid: "65549a4f1d274bada333aca776eeeba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce090c61274a4e50a4437f1269219a56" is_pre_election: true
I20260808 04:46:18.295908 5514 leader_election.cc:302] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3f45bbe328493682e6f9bee70382e2, 65549a4f1d274bada333aca776eeeba7; no voters:
I20260808 04:46:18.295967 5796 raft_consensus.cc:2454] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65549a4f1d274bada333aca776eeeba7 in term 0.
I20260808 04:46:18.296236 5899 raft_consensus.cc:2781] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:46:18.296360 5899 raft_consensus.cc:482] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:46:18.296427 5899 raft_consensus.cc:3037] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:46:18.297525 5899 raft_consensus.cc:504] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.297884 5899 leader_election.cc:288] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [CANDIDATE]: Term 1 election: Requested vote from peers 5a3f45bbe328493682e6f9bee70382e2 (127.22.2.129:44253), ce090c61274a4e50a4437f1269219a56 (127.22.2.131:46067)
I20260808 04:46:18.298264 5796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e88e6f5f204caaa44ca53332bda8a2" candidate_uuid: "65549a4f1d274bada333aca776eeeba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce090c61274a4e50a4437f1269219a56"
I20260808 04:46:18.298265 5403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e88e6f5f204caaa44ca53332bda8a2" candidate_uuid: "65549a4f1d274bada333aca776eeeba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f45bbe328493682e6f9bee70382e2"
I20260808 04:46:18.298398 5796 raft_consensus.cc:3037] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:46:18.298444 5403 raft_consensus.cc:3037] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:46:18.299516 5403 raft_consensus.cc:2454] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65549a4f1d274bada333aca776eeeba7 in term 1.
I20260808 04:46:18.299530 5796 raft_consensus.cc:2454] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65549a4f1d274bada333aca776eeeba7 in term 1.
I20260808 04:46:18.299816 5510 leader_election.cc:302] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65549a4f1d274bada333aca776eeeba7, ce090c61274a4e50a4437f1269219a56; no voters:
I20260808 04:46:18.300017 5899 raft_consensus.cc:2781] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:46:18.300248 5899 raft_consensus.cc:686] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 1 LEADER]: Becoming Leader. State: Replica: 65549a4f1d274bada333aca776eeeba7, State: Running, Role: LEADER
I20260808 04:46:18.300637 5899 consensus_queue.cc:237] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } }
I20260808 04:46:18.304119 5164 catalog_manager.cc:5495] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65549a4f1d274bada333aca776eeeba7 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "65549a4f1d274bada333aca776eeeba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65549a4f1d274bada333aca776eeeba7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 } health_report { overall_health: UNKNOWN } } }
W20260808 04:46:18.331043 5860 tablet.cc:2369] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:46:18.341205 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.337281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49816 (local address 127.22.2.130:39657)
0808 04:46:18.337404 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 04:46:18.337411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:46:18.337428 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:46:18.337664 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:46:18.337667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:46:18.337737 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:46:18.337849 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:46:18.338121 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.338703 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.339590 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:46:18.339823 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:46:18.339939 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:46:18.340268 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:46:18.340736 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:46:18.340901 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 04:46:18.340987 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 04:46:18.341045 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 04:46:18.347504 5403 raft_consensus.cc:1261] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [term 1 FOLLOWER]: Refusing update from remote peer 65549a4f1d274bada333aca776eeeba7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:46:18.347630 5796 raft_consensus.cc:1261] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 1 FOLLOWER]: Refusing update from remote peer 65549a4f1d274bada333aca776eeeba7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:46:18.348390 5899 consensus_queue.cc:1035] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3f45bbe328493682e6f9bee70382e2" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:46:18.348647 5903 consensus_queue.cc:1035] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce090c61274a4e50a4437f1269219a56" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 46067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:46:18.358471 5907 mvcc.cc:205] Tried to move back new op lower bound from 7316129293703737344 to 7316129293522251776. Current Snapshot: MvccSnapshot[applied={T|T < 7316129293703737344 or (T in {7316129293703737344})}]
I20260808 04:46:18.359145 5909 mvcc.cc:205] Tried to move back new op lower bound from 7316129293703737344 to 7316129293522251776. Current Snapshot: MvccSnapshot[applied={T|T < 7316129293703737344}]
I20260808 04:46:18.365329 5167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:46:18.365504 5167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:46:18.367903 5167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d718bd486d5649cdb04e7b03b341dbd3: Sending DeleteTablet for 3 replicas of tablet 46e88e6f5f204caaa44ca53332bda8a2
I20260808 04:46:18.368722 5367 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e88e6f5f204caaa44ca53332bda8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:46:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:41992
I20260808 04:46:18.368752 5580 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e88e6f5f204caaa44ca53332bda8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:46:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:49808
I20260808 04:46:18.368894 5767 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e88e6f5f204caaa44ca53332bda8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:46:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40952
I20260808 04:46:18.369323 5921 tablet_replica.cc:330] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: stopping tablet replica
I20260808 04:46:18.369663 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 5266
I20260808 04:46:18.369654 5920 tablet_replica.cc:330] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2: stopping tablet replica
I20260808 04:46:18.369673 5921 raft_consensus.cc:2229] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:46:18.369978 5920 raft_consensus.cc:2229] T 46e88e6f5f204caaa44ca53332bda8a2 P 5a3f45bbe328493682e6f9bee70382e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:46:18.369995 5921 raft_consensus.cc:2258] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:46:18.373661 5921 ts_tablet_manager.cc:1905] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:46:18.375888 5921 ts_tablet_manager.cc:1918] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:46:18.376016 5921 log.cc:1196] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/46e88e6f5f204caaa44ca53332bda8a2
I20260808 04:46:18.376340 5921 ts_tablet_manager.cc:1939] T 46e88e6f5f204caaa44ca53332bda8a2 P ce090c61274a4e50a4437f1269219a56: Deleting consensus metadata
I20260808 04:46:18.380805 5143 catalog_manager.cc:4841] TS ce090c61274a4e50a4437f1269219a56 (127.22.2.131:46067): tablet 46e88e6f5f204caaa44ca53332bda8a2 (table test-table [id=267ff3c4458a43b6b475d4acda10074c]) successfully deleted
I20260808 04:46:18.381701 5919 tablet_replica.cc:330] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: stopping tablet replica
I20260808 04:46:18.382055 5919 raft_consensus.cc:2229] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:46:18.382455 5919 raft_consensus.cc:2258] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:46:18.384059 5919 ts_tablet_manager.cc:1905] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:46:18.386950 5919 ts_tablet_manager.cc:1918] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:46:18.387056 5919 log.cc:1196] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/46e88e6f5f204caaa44ca53332bda8a2
I20260808 04:46:18.387389 5919 ts_tablet_manager.cc:1939] T 46e88e6f5f204caaa44ca53332bda8a2 P 65549a4f1d274bada333aca776eeeba7: Deleting consensus metadata
I20260808 04:46:18.388778 5137 catalog_manager.cc:4841] TS 65549a4f1d274bada333aca776eeeba7 (127.22.2.130:39657): tablet 46e88e6f5f204caaa44ca53332bda8a2 (table test-table [id=267ff3c4458a43b6b475d4acda10074c]) successfully deleted
W20260808 04:46:18.389145 5145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:44253 (error 108)
I20260808 04:46:18.389423 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 5477
I20260808 04:46:18.397523 5877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:46:18.396449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:44253 (local address 127.0.0.1:42006)
0808 04:46:18.397247 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 04:46:18.397381 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:44253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":669}
W20260808 04:46:18.398000 5145 catalog_manager.cc:4584] TS 5a3f45bbe328493682e6f9bee70382e2 (127.22.2.129:44253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 46e88e6f5f204caaa44ca53332bda8a2: Network error: Client connection negotiation failed: client connection to 127.22.2.129:44253: connect: Connection refused (error 111)
I20260808 04:46:18.403865 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 5679
I20260808 04:46:18.413753 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 5111
2026-08-08T04:46:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2685 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:46:18 dist-test-slave-04bb krb5kdc[5926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:46:18 dist-test-slave-04bb krb5kdc[5926](info): set up 2 sockets
Aug 08 04:46:18 dist-test-slave-04bb krb5kdc[5926](info): commencing operation
krb5kdc: starting...
W20260808 04:46:19.685230 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:46:19 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:46:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:46:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/rangeradmin_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/rangeradmin_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/rangerlookup_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/rangerlookup_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
I20260808 04:46:19.871973 22538 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzqcaaz/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/postgres -l logfile start
2026-08-08 04:46:22.483 UTC [5973] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 04:46:22.483 UTC [5973] LOG: listening on IPv4 address "127.22.2.148", port 34551
2026-08-08 04:46:22.487 UTC [5973] LOG: listening on Unix socket "/tmp/.s.PGSQL.34551"
2026-08-08 04:46:22.509 UTC [5977] LOG: database system was shut down at 2026-08-08 04:46:20 UTC
2026-08-08 04:46:22.513 UTC [5973] LOG: database system is ready to accept connections
I20260808 04:46:23.724984 22538 mini_postgres.cc:96] Postgres bound to 34551
2026-08-08 04:46:23.734 UTC [5991] FATAL: database "slave" does not exist
127.22.2.148:34551 - accepting connections
I20260808 04:46:23.734956 22538 mini_ranger.cc:161] Starting Ranger...
I20260808 04:46:23.751719 22538 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 04:46:24.260910 22538 mini_ranger.cc:87] Created ranger Postgres database
I20260808 04:46:24.261000 22538 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-admin
2026-08-08 04:46:24,587 [I] DB FLAVOR :POSTGRES
2026-08-08 04:46:24,588 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 04:46:24,588 [I] Checking connection..
2026-08-08 04:46:24,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 04:46:24,849 [I] Checking connection passed.
2026-08-08 04:46:24,850 [I] --------- Verifying version history table ---------
2026-08-08 04:46:24,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:46:25,109 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 04:46:25,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:46:25,358 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 04:46:25,359 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 04:46:25,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 04:46:25.596 UTC [6087] WARNING: there is no transaction in progress
2026-08-08 04:46:25,602 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 04:46:25,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:46:25,863 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 04:46:25,863 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 04:46:25,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 04:46:26,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 04:46:26,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-04bb.c.gcp-upstream.internal','N') ;"
2026-08-08 04:46:26,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 04:46:26,930 [I] Table x_portal_user does not exist in database ranger
2026-08-08 04:46:26,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 04:46:27,202 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 04:46:27,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 04:46:27,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 04:46:27,729 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 04:46:27,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 04:46:28.699 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.717 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.735 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.750 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.773 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.794 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.855 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.862 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.878 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.890 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.901 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.918 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.929 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.936 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.943 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:28.951 UTC [6237] WARNING: there is no transaction in progress
2026-08-08 04:46:29,635 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 04:46:29,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 04:46:29,896 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 04:46:29,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 04:46:30,160 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 04:46:30,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 04:46:30,393 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 04:46:30,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 04:46:30,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 04:46:30,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-04bb.c.gcp-upstream.internal';"
2026-08-08 04:46:31,181 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 04:46:31,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 04:46:31,446 [I] DB_PATCHES have already been applied
I20260808 04:46:31.452394 22538 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzqcaaz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 04:46:31.452490 22538 mini_ranger.cc:193] Using host: 127.22.2.148
I20260808 04:46:31.457116 22538 mini_ranger.cc:227] Ranger admin URL: http://127.22.2.148:54285
Aug 08, 2026 4:46:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:46:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:46:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 4:46:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 4:46:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 4:46:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 4:46:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 04:46:32.805663 22538 mini_ranger.cc:160] Time spent starting Ranger: real 9.071s user 0.004s sys 0.008s
Aug 08, 2026 4:46:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 4:46:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 4:46:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 4:46:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 4:46:54 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 4:46:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 04:46:55.566363 22538 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/rangerkms_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/rangerkms_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.2.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.2.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 04:46:55 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164415, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 04:46:55.676702 22538 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 04:46:55.687839 22538 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 04:46:56.597219 22538 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 04:46:56.597326 22538 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms
I20260808 04:46:56.597456 22538 mini_ranger_kms.cc:227] Using postgres at 127.22.2.148:34551
2026-08-08 04:46:56,878 [I] DB FLAVOR :POSTGRES
2026-08-08 04:46:56,878 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 04:46:56,879 [I] Checking connection
2026-08-08 04:46:56,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 04:46:57,166 [I] connection success
2026-08-08 04:46:57,166 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 04:46:57,166 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 04:46:57,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 04:46:57,426 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 04:46:57,426 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 04:46:57,426 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 04:46:57,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:34551/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 04:46:57,686 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 04:46:58.190625 22538 mini_ranger_kms.cc:310] Created kms service
I20260808 04:46:58.426369 22538 mini_ranger_kms.cc:326] Created kudu user
I20260808 04:46:58.478346 22538 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 04:46:58 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164415, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:46:58.905874 22538 mini_ranger_kms.cc:384] Added ranger policy
I20260808 04:46:58.906044 22538 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 04:46:58.906073 22538 mini_ranger_kms.cc:240] Using host: 127.22.2.148
I20260808 04:46:58.911751 22538 mini_ranger_kms.cc:276] Ranger KMS PID: 6504
I20260808 04:46:58.911891 22538 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.2.148:39949
Aug 08, 2026 4:47:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:47:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:47:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 4:47:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 4:47:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 4:47:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 04:47:01 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786164421, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:01.722335 22538 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.046s user 0.014s sys 0.246s
I20260808 04:47:01.722484 22538 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 04:47:01.722596 22538 mini_ranger_kms.cc:401] 127.22.2.148:39949/kms/v1/keys
Aug 08 04:47:02 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164421, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
Aug 08 04:47:02 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164421, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
Aug 08 04:47:04 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:47:04.856011 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:43253
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:43253
--ranger_config_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:39949/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:47:04.997661 6578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:04.997917 6578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:04.997958 6578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:05.002064 6578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:47:05.002288 6578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:05.002321 6578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:05.002345 6578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:47:05.002369 6578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:47:05.008165 6578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.2.148:39949/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:43253
--ranger_config_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:43253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:05.009258 6578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:05.010262 6578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:05.019551 6588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:05.019619 6578 server_base.cc:1034] running on GCE node
W20260808 04:47:05.019629 6590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:05.019711 6592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:05.020437 6578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:05.021545 6578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:05.022710 6578 hybrid_clock.cc:648] HybridClock initialized: now 1786164425022694 us; error 34 us; skew 500 ppm
Aug 08 04:47:05 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164425, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:05.025867 6578 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:47:05.026691 6578 webserver.cc:460] Webserver started at http://127.22.2.190:36847/ using document root <none> and password file <none>
I20260808 04:47:05.027186 6578 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:05.027240 6578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:47:05.027401 6578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 04:47:05 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164425, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:05.096076 6578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "7146180d39f044b3a88b8c094aeaee0e"
format_stamp: "Formatted at 2026-08-08 04:47:05 on dist-test-slave-04bb"
server_key: "c9fce70864641388bc9cba3ddd012657"
server_key_iv: "c823967c8c89f68b81a3f61a661c5d5b"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:05.096822 6578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "7146180d39f044b3a88b8c094aeaee0e"
format_stamp: "Formatted at 2026-08-08 04:47:05 on dist-test-slave-04bb"
server_key: "c9fce70864641388bc9cba3ddd012657"
server_key_iv: "c823967c8c89f68b81a3f61a661c5d5b"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:05.101052 6578 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 04:47:05.117870 6603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:05.119527 6578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:47:05.119678 6578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "7146180d39f044b3a88b8c094aeaee0e"
format_stamp: "Formatted at 2026-08-08 04:47:05 on dist-test-slave-04bb"
server_key: "c9fce70864641388bc9cba3ddd012657"
server_key_iv: "c823967c8c89f68b81a3f61a661c5d5b"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:05.119812 6578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:05.141428 6578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:05.144922 6578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:05.145150 6578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:05.157253 6578 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:43253
I20260808 04:47:05.157403 6684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:43253 every 8 connection(s)
I20260808 04:47:05.158483 6578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:47:05.161827 6685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:47:05.165997 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 6578
I20260808 04:47:05.166143 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:47:05.166491 22538 external_mini_cluster.cc:1428] Setting key e3d6cd224e4e39a296b69017f72b0c7d
I20260808 04:47:05.168602 6685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e: Bootstrap starting.
I20260808 04:47:05.171871 6685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e: Neither blocks nor log segments found. Creating new log.
I20260808 04:47:05.173168 6685 log.cc:824] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e: Log is configured to *not* fsync() on all Append() calls
I20260808 04:47:05.175588 6685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e: No bootstrap required, opened a new log
Aug 08 04:47:05 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:05.181231 6685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7146180d39f044b3a88b8c094aeaee0e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43253 } }
I20260808 04:47:05.181598 6685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:47:05.181715 6685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7146180d39f044b3a88b8c094aeaee0e, State: Initialized, Role: FOLLOWER
I20260808 04:47:05.182217 6685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7146180d39f044b3a88b8c094aeaee0e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43253 } }
I20260808 04:47:05.182371 6685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:47:05.182457 6685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:47:05.182555 6685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:47:05.183507 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:05.168082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.22.2.190:43253)
0808 04:47:05.168688 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:05.168709 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:05.168755 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:47:05.170007 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:05.170033 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:05.170484 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:05.170898 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:05.171849 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:05.173786 (+ 1937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:05.174729 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:05.175258 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:05.175454 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:05.178141 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:05.178147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:05.178162 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:05.178188 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:05.179996 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:05.180557 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:05.180563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:05.180573 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:05.180686 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:05.180994 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:05.180998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:05.181000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:05.181401 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:05.181632 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:05.182303 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 04:47:05.182627 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":183,"threads_started":1}
I20260808 04:47:05.183658 6685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7146180d39f044b3a88b8c094aeaee0e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43253 } }
I20260808 04:47:05.184041 6685 leader_election.cc:302] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7146180d39f044b3a88b8c094aeaee0e; no voters:
I20260808 04:47:05.184383 6685 leader_election.cc:288] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:47:05.184500 6693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:47:05.184841 6693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [term 1 LEADER]: Becoming Leader. State: Replica: 7146180d39f044b3a88b8c094aeaee0e, State: Running, Role: LEADER
I20260808 04:47:05.185190 6693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7146180d39f044b3a88b8c094aeaee0e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43253 } }
I20260808 04:47:05.185762 6685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:47:05.186488 6693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7146180d39f044b3a88b8c094aeaee0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7146180d39f044b3a88b8c094aeaee0e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43253 } } }
I20260808 04:47:05.186621 6693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [sys.catalog]: This master's current role is: LEADER
I20260808 04:47:05.187014 6693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7146180d39f044b3a88b8c094aeaee0e. Latest consensus state: current_term: 1 leader_uuid: "7146180d39f044b3a88b8c094aeaee0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7146180d39f044b3a88b8c094aeaee0e" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43253 } } }
I20260808 04:47:05.187110 6693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e [sys.catalog]: This master's current role is: LEADER
I20260808 04:47:05.189141 6702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:47:05.189697 6685 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 04:47:05.192868 6702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:47:05.198150 6702 catalog_manager.cc:1347] Generated new cluster ID: a4fd6cab580145d88e1bc089e228e75e
I20260808 04:47:05.198262 6702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:47:05.230100 6702 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:47:05.232656 6702 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:47:05.242197 6702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e: Generated new TSK 0
I20260808 04:47:05.243062 6702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:47:06.040320 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:39949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43253
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:47:06.194674 6753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:06.194967 6753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:06.195056 6753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:06.199245 6753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:06.199322 6753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:06.199410 6753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:47:06.205081 6753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.2.148:39949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:06.206190 6753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:06.207278 6753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:06.219625 6760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:06.219952 6753 server_base.cc:1034] running on GCE node
W20260808 04:47:06.219874 6762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:06.219658 6759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:06.220630 6753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:06.221462 6753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:06.222681 6753 hybrid_clock.cc:648] HybridClock initialized: now 1786164426222650 us; error 46 us; skew 500 ppm
Aug 08 04:47:06 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:06.226820 6753 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:47:06.228118 6753 webserver.cc:460] Webserver started at http://127.22.2.129:33761/ using document root <none> and password file <none>
I20260808 04:47:06.229099 6753 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:06.229194 6753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:47:06.229431 6753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 04:47:06 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:06.247336 6753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "667c5cb223e04df0ac50c48aa5a94f16"
format_stamp: "Formatted at 2026-08-08 04:47:06 on dist-test-slave-04bb"
server_key: "c2726bc324ea9d779154dbb9125f3bf5"
server_key_iv: "65eda52437b0cdbf08f23bd160afc692"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:06.248554 6753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "667c5cb223e04df0ac50c48aa5a94f16"
format_stamp: "Formatted at 2026-08-08 04:47:06 on dist-test-slave-04bb"
server_key: "c2726bc324ea9d779154dbb9125f3bf5"
server_key_iv: "65eda52437b0cdbf08f23bd160afc692"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:06.253947 6753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 04:47:06.275452 6772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:06.277347 6753 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 04:47:06.277498 6753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "667c5cb223e04df0ac50c48aa5a94f16"
format_stamp: "Formatted at 2026-08-08 04:47:06 on dist-test-slave-04bb"
server_key: "c2726bc324ea9d779154dbb9125f3bf5"
server_key_iv: "65eda52437b0cdbf08f23bd160afc692"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:06.277633 6753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:06.307291 6753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:06.312672 6753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:06.312945 6753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:06.313941 6753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:06.315441 6753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:06.315594 6753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:06.315735 6753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:06.315819 6753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:06.336822 6753 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:39447
I20260808 04:47:06.338208 6753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:47:06.341447 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 6753
I20260808 04:47:06.341593 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:47:06.341989 22538 external_mini_cluster.cc:1428] Setting key e85841e90ec0b75dbb7ef193387511df
I20260808 04:47:06.345289 6932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:39447 every 8 connection(s)
Aug 08 04:47:06 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:06.364171 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:06.341700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43253 (local address 127.22.2.129:53821)
0808 04:47:06.345751 (+ 4051us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:06.345811 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:06.347153 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:06.349254 (+ 2101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:06.349264 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:06.349852 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:06.350647 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:06.350667 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:06.351890 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:06.351893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:06.353625 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:06.353639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:06.354476 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:06.354481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:06.354772 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:06.355722 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:06.355747 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:06.358109 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:06.360713 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:06.360720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:06.360738 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:06.361093 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:06.361441 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:06.361444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:06.361446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:06.361635 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:06.362155 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:06.362162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:06.362472 (+ 310us) client_negotiation.cc:769] Sending connection context
0808 04:47:06.362795 (+ 323us) client_negotiation.cc:240] Negotiation successful
0808 04:47:06.363136 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3794,"thread_start_us":208,"threads_started":1}
I20260808 04:47:06.365031 6939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:06.342928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:53821 (local address 127.22.2.190:43253)
0808 04:47:06.348727 (+ 5799us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:06.348733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:06.348747 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:47:06.348787 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:06.348791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:06.348837 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:06.348927 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:06.350831 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:06.351673 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:06.353797 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:06.354167 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:06.354278 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:06.358360 (+ 4082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:06.358366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:06.358369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:06.358403 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:06.360523 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:06.361227 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:06.361231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:06.361233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:06.361331 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:06.361757 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:06.361761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:06.361763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:06.361995 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:06.362095 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:06.364741 (+ 2646us) server_negotiation.cc:299] Negotiation successful
0808 04:47:06.364872 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5708,"thread_start_us":91,"threads_started":1}
I20260808 04:47:06.366780 6933 heartbeater.cc:347] Connected to a master server at 127.22.2.190:43253
I20260808 04:47:06.367048 6933 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:06.367640 6933 heartbeater.cc:511] Master 127.22.2.190:43253 requested a full tablet report, sending...
I20260808 04:47:06.369416 6626 ts_manager.cc:194] Registered new tserver with Master: 667c5cb223e04df0ac50c48aa5a94f16 (127.22.2.129:39447)
I20260808 04:47:06.370934 6626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:53821
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:47:06.431887 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:39949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43253
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:47:06 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 04:47:06.636884 6945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:06.637131 6945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:06.637202 6945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:06.641261 6945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:06.641314 6945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:06.641395 6945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:47:06.648792 6945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.2.148:39949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:06.650120 6945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:06.651485 6945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:06.665874 6956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:06.666402 6959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:06.666615 6957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:06.667212 6945 server_base.cc:1034] running on GCE node
I20260808 04:47:06.667891 6945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:06.668689 6945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:06.670004 6945 hybrid_clock.cc:648] HybridClock initialized: now 1786164426669966 us; error 49 us; skew 500 ppm
Aug 08 04:47:06 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:06.674816 6945 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:47:06.676323 6945 webserver.cc:460] Webserver started at http://127.22.2.130:42963/ using document root <none> and password file <none>
I20260808 04:47:06.677143 6945 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:06.677256 6945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:47:06.677585 6945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 04:47:06 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:06.698843 6945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "cd03e5c00f0b49b98f660849c99590fd"
format_stamp: "Formatted at 2026-08-08 04:47:06 on dist-test-slave-04bb"
server_key: "f3f820dff398f2e3b9a036ece98d73a7"
server_key_iv: "3e6cc6fdfdd1426b13fc23567bc5fb0b"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:06.699666 6945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "cd03e5c00f0b49b98f660849c99590fd"
format_stamp: "Formatted at 2026-08-08 04:47:06 on dist-test-slave-04bb"
server_key: "f3f820dff398f2e3b9a036ece98d73a7"
server_key_iv: "3e6cc6fdfdd1426b13fc23567bc5fb0b"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:06.706005 6945 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260808 04:47:06.732964 6969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:06.734951 6945 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260808 04:47:06.735176 6945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "cd03e5c00f0b49b98f660849c99590fd"
format_stamp: "Formatted at 2026-08-08 04:47:06 on dist-test-slave-04bb"
server_key: "f3f820dff398f2e3b9a036ece98d73a7"
server_key_iv: "3e6cc6fdfdd1426b13fc23567bc5fb0b"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:06.735441 6945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:06.774437 6945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:06.779176 6945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:06.779412 6945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:06.780161 6945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:06.781260 6945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:06.781330 6945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:06.781430 6945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:06.781473 6945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:06.796361 6945 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:46343
I20260808 04:47:06.797794 6945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:47:06.800064 7138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:46343 every 8 connection(s)
I20260808 04:47:06.806581 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 6945
I20260808 04:47:06.806739 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:47:06.807058 22538 external_mini_cluster.cc:1428] Setting key d9d20af5d9b2d8c9938a1cc6c3a7598d
Aug 08 04:47:06 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:06.819408 6939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:06.800646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53493 (local address 127.22.2.190:43253)
0808 04:47:06.800789 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:06.800793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:06.801737 (+ 944us) server_negotiation.cc:407] Connection header received
0808 04:47:06.802929 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:06.802933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:06.802996 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:06.803105 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:06.804617 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:06.805463 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:06.807597 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:06.808049 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:06.808171 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:06.811841 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:06.811853 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:06.811857 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:06.811894 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:06.814552 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:06.815982 (+ 1430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:06.815986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:06.815996 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:06.816134 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:06.816760 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:06.816776 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:06.816778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:06.817171 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:06.817286 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:06.818894 (+ 1608us) server_negotiation.cc:299] Negotiation successful
0808 04:47:06.819179 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:47:06.821966 7141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:06.800579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43253 (local address 127.22.2.130:53493)
0808 04:47:06.801540 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:06.801597 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:06.802646 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:06.803323 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:06.803333 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:06.803697 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:06.804401 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:06.804422 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:06.805614 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:06.805617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:06.807406 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:06.807426 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:06.808441 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:06.808445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:06.808741 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:06.809541 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:06.809571 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:06.811565 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:06.814751 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:06.814758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:06.814778 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:06.815761 (+ 983us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:06.816343 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:06.816348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:06.816351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:06.816604 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:06.817442 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:06.817449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:06.818382 (+ 933us) client_negotiation.cc:769] Sending connection context
0808 04:47:06.819667 (+ 1285us) client_negotiation.cc:240] Negotiation successful
0808 04:47:06.820224 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":142,"threads_started":1}
I20260808 04:47:06.823752 7139 heartbeater.cc:347] Connected to a master server at 127.22.2.190:43253
I20260808 04:47:06.824071 7139 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:47:06.825063 7139 heartbeater.cc:511] Master 127.22.2.190:43253 requested a full tablet report, sending...
I20260808 04:47:06.827570 6626 ts_manager.cc:194] Registered new tserver with Master: cd03e5c00f0b49b98f660849c99590fd (127.22.2.130:46343)
I20260808 04:47:06.828341 6626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:53493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:47:06.865056 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:39949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43253
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:47:07 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
W20260808 04:47:07.069056 7145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:07.069514 7145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:07.069645 7145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:07.075613 7145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:07.075690 7145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:07.075774 7145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:47:07.083287 7145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.2.148:39949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:07.084666 7145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:07.086261 7145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:07.102952 7150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:07.103051 7153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:07.102952 7151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:07.104077 7145 server_base.cc:1034] running on GCE node
I20260808 04:47:07.104513 7145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:07.105268 7145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:07.106480 7145 hybrid_clock.cc:648] HybridClock initialized: now 1786164427106404 us; error 87 us; skew 500 ppm
Aug 08 04:47:07 dist-test-slave-04bb krb5kdc[5926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164427, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:07.110996 7145 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:47:07.112090 7145 webserver.cc:460] Webserver started at http://127.22.2.131:35675/ using document root <none> and password file <none>
I20260808 04:47:07.112920 7145 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:07.113071 7145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:47:07.113363 7145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 04:47:07 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164427, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:07.134487 7145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "440af54bf2cc4c14805768c2fea2a7e3"
format_stamp: "Formatted at 2026-08-08 04:47:07 on dist-test-slave-04bb"
server_key: "39e7174366b61f1a66bb1081d406bffc"
server_key_iv: "1e2127645bbe7a7a495531f406d80ad4"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:07.135469 7145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "440af54bf2cc4c14805768c2fea2a7e3"
format_stamp: "Formatted at 2026-08-08 04:47:07 on dist-test-slave-04bb"
server_key: "39e7174366b61f1a66bb1081d406bffc"
server_key_iv: "1e2127645bbe7a7a495531f406d80ad4"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:07.140993 7145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:47:07.165105 7162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:07.167086 7145 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.000s
I20260808 04:47:07.167248 7145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "440af54bf2cc4c14805768c2fea2a7e3"
format_stamp: "Formatted at 2026-08-08 04:47:07 on dist-test-slave-04bb"
server_key: "39e7174366b61f1a66bb1081d406bffc"
server_key_iv: "1e2127645bbe7a7a495531f406d80ad4"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:07.167404 7145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:07.216260 7145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:07.220419 7145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:07.220676 7145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:07.221405 7145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:07.222487 7145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:07.222561 7145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:07.222674 7145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:07.222728 7145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:07.242300 7145 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:40253
I20260808 04:47:07.243170 7327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:40253 every 8 connection(s)
I20260808 04:47:07.243575 7145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:47:07.252710 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 7145
I20260808 04:47:07.252854 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:47:07.253209 22538 external_mini_cluster.cc:1428] Setting key 13cd3d694c9c35304c913aabfe2c95d6
Aug 08 04:47:07 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164427, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:07.261298 6939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.246537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:43513 (local address 127.22.2.190:43253)
0808 04:47:07.246700 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:07.246705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:07.249066 (+ 2361us) server_negotiation.cc:407] Connection header received
0808 04:47:07.250273 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:07.250276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:07.250343 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:07.250470 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:07.251616 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.252204 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.254067 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.254425 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.254553 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.257365 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:07.257369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:07.257372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:07.257399 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:07.258987 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:07.259551 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:07.259555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:07.259557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:07.259641 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:07.260028 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:07.260032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:07.260034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:07.260236 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:07.260349 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:07.260970 (+ 621us) server_negotiation.cc:299] Negotiation successful
0808 04:47:07.261119 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 04:47:07.262188 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.247051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43253 (local address 127.22.2.131:43513)
0808 04:47:07.248895 (+ 1844us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:07.248941 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:07.250034 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:07.250663 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:07.250671 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:07.250956 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:07.251459 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.251473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.252338 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.252343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.253888 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.253898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.254591 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.254595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.254807 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.255517 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:07.255544 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:07.257198 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:07.259128 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:07.259136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:07.259151 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:07.259420 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:07.259744 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:07.259747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:07.259749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:07.259913 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:07.260348 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:07.260355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:07.260713 (+ 358us) client_negotiation.cc:769] Sending connection context
0808 04:47:07.260905 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 04:47:07.261222 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1600,"thread_start_us":243,"threads_started":1}
I20260808 04:47:07.263550 7328 heartbeater.cc:347] Connected to a master server at 127.22.2.190:43253
I20260808 04:47:07.263823 7328 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:07.264319 7328 heartbeater.cc:511] Master 127.22.2.190:43253 requested a full tablet report, sending...
I20260808 04:47:07.265447 6626 ts_manager.cc:194] Registered new tserver with Master: 440af54bf2cc4c14805768c2fea2a7e3 (127.22.2.131:40253)
I20260808 04:47:07.266263 6626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:43513
I20260808 04:47:07.268891 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:47:07.282148 6939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.272951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41942 (local address 127.22.2.190:43253)
0808 04:47:07.273071 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:07.273078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:07.273796 (+ 718us) server_negotiation.cc:407] Connection header received
0808 04:47:07.273943 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:07.273946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:07.273994 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:07.274063 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:07.275029 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.275535 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.276403 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.276680 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.276784 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.278482 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:07.278486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:07.278489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:07.278518 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:07.280190 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:07.280730 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:07.280735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:07.280739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:07.280833 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:07.281159 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:07.281163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:07.281165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:07.281359 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:07.281507 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:07.281772 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 04:47:07.281893 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 04:47:07.285436 6626 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:47:07.289227 6626 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:47:07.315460 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.303762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40253 (local address 127.0.0.1:34268)
0808 04:47:07.306284 (+ 2522us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:07.306299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:07.306521 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:07.308982 (+ 2461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:07.308985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:07.309002 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:07.309124 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.309131 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.310606 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.310610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.311966 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.311979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.314864 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.314869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.314987 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.315034 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 04:47:07.315174 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 04:47:07.315296 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2458,"thread_start_us":1406,"threads_started":1}
I20260808 04:47:07.319026 7349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.303933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.22.2.131:40253)
0808 04:47:07.306342 (+ 2409us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:07.306361 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:07.306624 (+ 263us) server_negotiation.cc:407] Connection header received
0808 04:47:07.306710 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:07.306715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:07.306945 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:07.307335 (+ 390us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:07.309274 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.310452 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.313620 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.314517 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.314637 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.314794 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:07.318677 (+ 3883us) server_negotiation.cc:299] Negotiation successful
0808 04:47:07.318792 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2300,"thread_start_us":266,"threads_started":1}
I20260808 04:47:07.319510 7355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.305848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40060 (local address 127.22.2.130:46343)
0808 04:47:07.307613 (+ 1765us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:07.307658 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:07.307682 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:47:07.307759 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:07.307764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:07.307967 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:07.308461 (+ 494us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:07.312150 (+ 3689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.313346 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.317400 (+ 4054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.318327 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.318445 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.318590 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:07.319185 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 04:47:07.319308 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1664,"thread_start_us":135,"threads_started":1}
I20260808 04:47:07.319068 7353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.305369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:46343 (local address 127.0.0.1:40060)
0808 04:47:07.306540 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:07.306553 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:07.306785 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:07.308648 (+ 1863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:07.308651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:07.308671 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:07.308849 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.308857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.315855 (+ 6998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.315859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.317142 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.317155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.318474 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.318480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.318647 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.318689 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 04:47:07.318784 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 04:47:07.318875 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"thread_start_us":135,"threads_started":1}
I20260808 04:47:07.323251 7037 tablet_service.cc:1467] Processing CreateTablet for tablet 20a48a37ac28463584d3b279717520c0 (DEFAULT_TABLE table=test-table [id=5e5a1b0b2e5e49cda70c6883b403d78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:47:07.324553 7037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a48a37ac28463584d3b279717520c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:47:07.325229 7352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.303728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:39447 (local address 127.0.0.1:44942)
0808 04:47:07.305622 (+ 1894us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:07.305673 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:07.305918 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:07.308282 (+ 2364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:07.308288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:07.308313 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:07.308450 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.308458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.310823 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.310827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.311987 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.312000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.324810 (+ 12810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.324814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.324913 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.324940 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 04:47:07.324998 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 04:47:07.325088 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1682,"spinlock_wait_cycles":7424,"thread_start_us":694,"threads_started":1}
I20260808 04:47:07.326608 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.303858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.22.2.129:39447)
0808 04:47:07.307758 (+ 3900us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:07.307765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:07.307790 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:47:07.307887 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:07.307892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:07.308088 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:07.308641 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:07.308781 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.309967 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.312155 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.324637 (+ 12482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.326132 (+ 1495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.326304 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:07.326362 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 04:47:07.326464 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3788,"thread_start_us":79,"threads_started":1}
I20260808 04:47:07.329823 6826 tablet_service.cc:1467] Processing CreateTablet for tablet 20a48a37ac28463584d3b279717520c0 (DEFAULT_TABLE table=test-table [id=5e5a1b0b2e5e49cda70c6883b403d78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:47:07.331090 6826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a48a37ac28463584d3b279717520c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:47:07.339615 7231 tablet_service.cc:1467] Processing CreateTablet for tablet 20a48a37ac28463584d3b279717520c0 (DEFAULT_TABLE table=test-table [id=5e5a1b0b2e5e49cda70c6883b403d78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:47:07.340813 7231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a48a37ac28463584d3b279717520c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:47:07.341320 7357 tablet_bootstrap.cc:492] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16: Bootstrap starting.
I20260808 04:47:07.345146 7357 tablet_bootstrap.cc:654] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16: Neither blocks nor log segments found. Creating new log.
I20260808 04:47:07.346204 7357 log.cc:824] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16: Log is configured to *not* fsync() on all Append() calls
I20260808 04:47:07.348869 7357 tablet_bootstrap.cc:492] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16: No bootstrap required, opened a new log
I20260808 04:47:07.349092 7357 ts_tablet_manager.cc:1397] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 04:47:07.354103 7359 tablet_bootstrap.cc:492] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Bootstrap starting.
I20260808 04:47:07.355192 7357 raft_consensus.cc:348] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.355611 7357 raft_consensus.cc:374] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:47:07.355731 7357 raft_consensus.cc:729] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667c5cb223e04df0ac50c48aa5a94f16, State: Initialized, Role: FOLLOWER
I20260808 04:47:07.356312 7357 consensus_queue.cc:260] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.357416 7362 tablet_bootstrap.cc:492] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd: Bootstrap starting.
I20260808 04:47:07.358147 7357 ts_tablet_manager.cc:1428] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16: Time spent starting tablet: real 0.009s user 0.002s sys 0.005s
I20260808 04:47:07.358826 6933 heartbeater.cc:503] Master 127.22.2.190:43253 was elected leader, sending a full tablet report...
I20260808 04:47:07.359977 7362 tablet_bootstrap.cc:654] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd: Neither blocks nor log segments found. Creating new log.
I20260808 04:47:07.361204 7359 tablet_bootstrap.cc:654] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Neither blocks nor log segments found. Creating new log.
I20260808 04:47:07.362643 7359 log.cc:824] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Log is configured to *not* fsync() on all Append() calls
I20260808 04:47:07.364569 7362 log.cc:824] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd: Log is configured to *not* fsync() on all Append() calls
I20260808 04:47:07.365581 7359 tablet_bootstrap.cc:492] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: No bootstrap required, opened a new log
I20260808 04:47:07.365856 7359 ts_tablet_manager.cc:1397] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.003s
I20260808 04:47:07.366875 7362 tablet_bootstrap.cc:492] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd: No bootstrap required, opened a new log
I20260808 04:47:07.367082 7362 ts_tablet_manager.cc:1397] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260808 04:47:07.372826 7359 raft_consensus.cc:348] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.373200 7359 raft_consensus.cc:374] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:47:07.373298 7359 raft_consensus.cc:729] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 440af54bf2cc4c14805768c2fea2a7e3, State: Initialized, Role: FOLLOWER
I20260808 04:47:07.373364 7362 raft_consensus.cc:348] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.373711 7362 raft_consensus.cc:374] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:47:07.373781 7362 raft_consensus.cc:729] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd03e5c00f0b49b98f660849c99590fd, State: Initialized, Role: FOLLOWER
I20260808 04:47:07.374058 7359 consensus_queue.cc:260] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.374485 7362 consensus_queue.cc:260] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.375398 7359 ts_tablet_manager.cc:1428] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260808 04:47:07.375838 7328 heartbeater.cc:503] Master 127.22.2.190:43253 was elected leader, sending a full tablet report...
I20260808 04:47:07.375993 7362 ts_tablet_manager.cc:1428] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260808 04:47:07.376725 7139 heartbeater.cc:503] Master 127.22.2.190:43253 was elected leader, sending a full tablet report...
Aug 08 04:47:07 dist-test-slave-04bb krb5kdc[5926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164426, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:07.435858 7363 raft_consensus.cc:482] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:47:07.436089 7363 raft_consensus.cc:504] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.437515 7363 leader_election.cc:288] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd03e5c00f0b49b98f660849c99590fd (127.22.2.130:46343), 440af54bf2cc4c14805768c2fea2a7e3 (127.22.2.131:40253)
I20260808 04:47:07.445173 7355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.439198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:38263 (local address 127.22.2.130:46343)
0808 04:47:07.439458 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:07.439463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:07.439887 (+ 424us) server_negotiation.cc:407] Connection header received
0808 04:47:07.440100 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:07.440104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:07.440153 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:07.440200 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:07.440784 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.441624 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.443536 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.444225 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.444355 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.444416 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:07.444929 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 04:47:07.445015 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260808 04:47:07.446791 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.437964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:46343 (local address 127.22.2.129:38263)
0808 04:47:07.439736 (+ 1772us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:07.439751 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:07.439976 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:07.440340 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:07.440343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:07.440358 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:07.440595 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.440603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.441793 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.441796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.443394 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.443406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.444613 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.444619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.444773 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.444810 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 04:47:07.445736 (+ 926us) client_negotiation.cc:240] Negotiation successful
0808 04:47:07.445815 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1694,"thread_start_us":250,"threads_started":1}
I20260808 04:47:07.447855 7062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a48a37ac28463584d3b279717520c0" candidate_uuid: "667c5cb223e04df0ac50c48aa5a94f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd03e5c00f0b49b98f660849c99590fd" is_pre_election: true
I20260808 04:47:07.448221 7062 raft_consensus.cc:2454] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 667c5cb223e04df0ac50c48aa5a94f16 in term 0.
I20260808 04:47:07.449088 6778 leader_election.cc:302] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 667c5cb223e04df0ac50c48aa5a94f16, cd03e5c00f0b49b98f660849c99590fd; no voters:
I20260808 04:47:07.449404 7363 raft_consensus.cc:2781] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:47:07.449474 7363 raft_consensus.cc:482] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:47:07.449513 7363 raft_consensus.cc:3037] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:47:07.451288 7363 raft_consensus.cc:504] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.451653 7363 leader_election.cc:288] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [CANDIDATE]: Term 1 election: Requested vote from peers cd03e5c00f0b49b98f660849c99590fd (127.22.2.130:46343), 440af54bf2cc4c14805768c2fea2a7e3 (127.22.2.131:40253)
I20260808 04:47:07.452266 7062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a48a37ac28463584d3b279717520c0" candidate_uuid: "667c5cb223e04df0ac50c48aa5a94f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd03e5c00f0b49b98f660849c99590fd"
I20260808 04:47:07.452389 7062 raft_consensus.cc:3037] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:47:07.453946 7062 raft_consensus.cc:2454] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 667c5cb223e04df0ac50c48aa5a94f16 in term 1.
I20260808 04:47:07.454352 6778 leader_election.cc:302] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 667c5cb223e04df0ac50c48aa5a94f16, cd03e5c00f0b49b98f660849c99590fd; no voters:
I20260808 04:47:07.456061 7363 raft_consensus.cc:2781] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:47:07.456409 7363 raft_consensus.cc:686] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [term 1 LEADER]: Becoming Leader. State: Replica: 667c5cb223e04df0ac50c48aa5a94f16, State: Running, Role: LEADER
I20260808 04:47:07.456784 7349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.439321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:57843 (local address 127.22.2.131:40253)
0808 04:47:07.439549 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:07.439554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:07.444632 (+ 5078us) server_negotiation.cc:407] Connection header received
0808 04:47:07.444870 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:07.444876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:07.444941 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:07.444988 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:07.445646 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.446537 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.455245 (+ 8708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.455856 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.455971 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.456039 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:07.456573 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 04:47:07.456660 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260808 04:47:07.456840 7363 consensus_queue.cc:237] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } }
I20260808 04:47:07.457255 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.438529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40253 (local address 127.22.2.129:57843)
0808 04:47:07.444423 (+ 5894us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:07.444440 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:07.444720 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:07.445120 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:07.445122 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:07.445139 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:07.445491 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.445504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.446847 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.446850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.448185 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:07.448196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.456152 (+ 7956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.456155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:07.456378 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.456411 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 04:47:07.456982 (+ 571us) client_negotiation.cc:240] Negotiation successful
0808 04:47:07.457063 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5821,"thread_start_us":292,"threads_started":1}
I20260808 04:47:07.457917 7261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a48a37ac28463584d3b279717520c0" candidate_uuid: "667c5cb223e04df0ac50c48aa5a94f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "440af54bf2cc4c14805768c2fea2a7e3" is_pre_election: true
I20260808 04:47:07.458289 7261 raft_consensus.cc:2454] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 667c5cb223e04df0ac50c48aa5a94f16 in term 0.
I20260808 04:47:07.458689 7259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a48a37ac28463584d3b279717520c0" candidate_uuid: "667c5cb223e04df0ac50c48aa5a94f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "440af54bf2cc4c14805768c2fea2a7e3"
I20260808 04:47:07.458794 7259 raft_consensus.cc:3037] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:47:07.460942 7259 raft_consensus.cc:2454] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 667c5cb223e04df0ac50c48aa5a94f16 in term 1.
I20260808 04:47:07.473564 6625 catalog_manager.cc:5495] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 667c5cb223e04df0ac50c48aa5a94f16 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "667c5cb223e04df0ac50c48aa5a94f16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667c5cb223e04df0ac50c48aa5a94f16" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 } health_report { overall_health: UNKNOWN } } }
W20260808 04:47:07.496412 7330 tablet.cc:2369] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:47:07.514249 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.509022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44952 (local address 127.22.2.129:39447)
0808 04:47:07.509146 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:07.509150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:07.509165 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:47:07.509248 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:07.509250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:07.509308 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:07.509427 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:47:07.509722 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.510524 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.511814 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:07.512167 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:07.512294 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:07.512789 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:47:07.513424 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:47:07.513556 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:07.513993 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 04:47:07.514059 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 04:47:07.536291 7259 raft_consensus.cc:1261] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 1 FOLLOWER]: Refusing update from remote peer 667c5cb223e04df0ac50c48aa5a94f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:47:07.537389 7062 raft_consensus.cc:1261] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 1 FOLLOWER]: Refusing update from remote peer 667c5cb223e04df0ac50c48aa5a94f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:47:07.538467 7375 consensus_queue.cc:1035] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "440af54bf2cc4c14805768c2fea2a7e3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:47:07.539064 7363 consensus_queue.cc:1035] T 20a48a37ac28463584d3b279717520c0 P 667c5cb223e04df0ac50c48aa5a94f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd03e5c00f0b49b98f660849c99590fd" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 04:47:07.552819 7140 tablet.cc:2369] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:47:07.580518 6626 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:47:07.580744 6626 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:47:07.583595 6626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7146180d39f044b3a88b8c094aeaee0e: Sending DeleteTablet for 3 replicas of tablet 20a48a37ac28463584d3b279717520c0
I20260808 04:47:07.585428 7037 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a48a37ac28463584d3b279717520c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:47:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40060
I20260808 04:47:07.586133 7231 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a48a37ac28463584d3b279717520c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:47:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:34268
I20260808 04:47:07.587414 6826 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a48a37ac28463584d3b279717520c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:47:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:44942
I20260808 04:47:07.587572 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 6753
I20260808 04:47:07.600934 7390 tablet_replica.cc:330] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd: stopping tablet replica
I20260808 04:47:07.601399 7390 raft_consensus.cc:2229] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:47:07.601838 7390 raft_consensus.cc:2258] T 20a48a37ac28463584d3b279717520c0 P cd03e5c00f0b49b98f660849c99590fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:47:07.602671 7391 tablet_replica.cc:330] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: stopping tablet replica
I20260808 04:47:07.603015 7391 raft_consensus.cc:2229] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:47:07.603345 7391 raft_consensus.cc:2258] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 04:47:07.606601 6607 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:39447 (error 108)
I20260808 04:47:07.606876 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 6945
I20260808 04:47:07.618578 7391 ts_tablet_manager.cc:1905] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:47:07.621706 7391 ts_tablet_manager.cc:1918] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:47:07.621826 7391 log.cc:1196] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/20a48a37ac28463584d3b279717520c0
I20260808 04:47:07.622193 7391 ts_tablet_manager.cc:1939] T 20a48a37ac28463584d3b279717520c0 P 440af54bf2cc4c14805768c2fea2a7e3: Deleting consensus metadata
I20260808 04:47:07.623728 6605 catalog_manager.cc:4841] TS 440af54bf2cc4c14805768c2fea2a7e3 (127.22.2.131:40253): tablet 20a48a37ac28463584d3b279717520c0 (table test-table [id=5e5a1b0b2e5e49cda70c6883b403d78f]) successfully deleted
I20260808 04:47:07.625061 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:07.624685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:39447 (local address 127.0.0.1:44958)
0808 04:47:07.624876 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:07.624952 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260808 04:47:07.625339 6607 catalog_manager.cc:4584] TS 667c5cb223e04df0ac50c48aa5a94f16 (127.22.2.129:39447): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20a48a37ac28463584d3b279717520c0: Network error: Client connection negotiation failed: client connection to 127.22.2.129:39447: connect: Connection refused (error 111)
W20260808 04:47:07.625984 7354 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:47:07.625730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:46343 (local address 127.0.0.1:40076)
0808 04:47:07.625856 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:07.625911 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.130:46343: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 04:47:07.626350 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 7145
I20260808 04:47:07.644467 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 6578
I20260808 04:47:07.666234 22538 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 04:47:08.011137 22538 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 04:47:08.011417 22538 mini_ranger.cc:66] Stopping Ranger...
I20260808 04:47:08.355806 22538 mini_ranger.cc:68] Stopped Ranger
2026-08-08 04:47:08.356 UTC [5973] LOG: received smart shutdown request
2026-08-08 04:47:08.359 UTC [5973] LOG: background worker "logical replication launcher" (PID 5987) exited with exit code 1
2026-08-08 04:47:08.360 UTC [5978] LOG: shutting down
2026-08-08 04:47:08.520 UTC [5973] LOG: database system is shut down
2026-08-08T04:47:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50238 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:47:08 dist-test-slave-04bb krb5kdc[7406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:47:08 dist-test-slave-04bb krb5kdc[7406](info): set up 2 sockets
Aug 08 04:47:08 dist-test-slave-04bb krb5kdc[7406](info): commencing operation
krb5kdc: starting...
W20260808 04:47:09.949102 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:47:10 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:47:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/rangeradmin_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/rangeradmin_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/rangerlookup_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/rangerlookup_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
I20260808 04:47:10.127699 22538 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzqcaaz/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/postgres -l logfile start
2026-08-08 04:47:12.621 UTC [7449] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 04:47:12.621 UTC [7449] LOG: listening on IPv4 address "127.22.2.148", port 35251
2026-08-08 04:47:12.624 UTC [7449] LOG: listening on Unix socket "/tmp/.s.PGSQL.35251"
2026-08-08 04:47:12.646 UTC [7454] LOG: database system was shut down at 2026-08-08 04:47:11 UTC
2026-08-08 04:47:12.650 UTC [7449] LOG: database system is ready to accept connections
I20260808 04:47:13.836216 22538 mini_postgres.cc:96] Postgres bound to 35251
2026-08-08 04:47:13.843 UTC [7464] FATAL: database "slave" does not exist
127.22.2.148:35251 - accepting connections
I20260808 04:47:13.844310 22538 mini_ranger.cc:161] Starting Ranger...
I20260808 04:47:13.860275 22538 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 04:47:14.362848 22538 mini_ranger.cc:87] Created ranger Postgres database
I20260808 04:47:14.362926 22538 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-admin
2026-08-08 04:47:14,648 [I] DB FLAVOR :POSTGRES
2026-08-08 04:47:14,649 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 04:47:14,649 [I] Checking connection..
2026-08-08 04:47:14,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 04:47:14,910 [I] Checking connection passed.
2026-08-08 04:47:14,910 [I] --------- Verifying version history table ---------
2026-08-08 04:47:14,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:47:15,172 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 04:47:15,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:47:15,433 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 04:47:15,434 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 04:47:15,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 04:47:15.686 UTC [7550] WARNING: there is no transaction in progress
2026-08-08 04:47:15,694 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 04:47:15,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:47:15,950 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 04:47:15,951 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 04:47:15,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 04:47:16,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 04:47:16,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-04bb.c.gcp-upstream.internal','N') ;"
2026-08-08 04:47:16,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 04:47:17,008 [I] Table x_portal_user does not exist in database ranger
2026-08-08 04:47:17,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 04:47:17,275 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 04:47:17,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 04:47:17,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 04:47:17,764 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 04:47:17,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 04:47:18.808 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.826 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.843 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.860 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.878 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.897 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.966 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.973 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.988 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:18.999 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:19.009 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:19.023 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:19.032 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:19.038 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:19.045 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:19.052 UTC [7707] WARNING: there is no transaction in progress
2026-08-08 04:47:19,723 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 04:47:19,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 04:47:19,990 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 04:47:19,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 04:47:20,240 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 04:47:20,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 04:47:20,529 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 04:47:20,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 04:47:20,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 04:47:21,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-04bb.c.gcp-upstream.internal';"
2026-08-08 04:47:21,276 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 04:47:21,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 04:47:21,541 [I] DB_PATCHES have already been applied
I20260808 04:47:21.547515 22538 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzqcaaz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 04:47:21.547582 22538 mini_ranger.cc:193] Using host: 127.22.2.148
I20260808 04:47:21.552184 22538 mini_ranger.cc:227] Ranger admin URL: http://127.22.2.148:54911
Aug 08, 2026 4:47:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:47:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:47:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 4:47:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 4:47:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 4:47:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 4:47:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 04:47:23.001499 22538 mini_ranger.cc:160] Time spent starting Ranger: real 9.157s user 0.000s sys 0.012s
Aug 08, 2026 4:47:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 4:47:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 4:47:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 4:47:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 4:47:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 04:47:44.667631 22538 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/rangerkms_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/rangerkms_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.2.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.2.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 04:47:44 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164464, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 04:47:44.757131 22538 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 04:47:44.767314 22538 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 04:47:45.679782 22538 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 04:47:45.679857 22538 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms
I20260808 04:47:45.679870 22538 mini_ranger_kms.cc:227] Using postgres at 127.22.2.148:35251
2026-08-08 04:47:45,858 [I] DB FLAVOR :POSTGRES
2026-08-08 04:47:45,859 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 04:47:45,859 [I] Checking connection
2026-08-08 04:47:45,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 04:47:46,106 [I] connection success
2026-08-08 04:47:46,107 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 04:47:46,107 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 04:47:46,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 04:47:46,349 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 04:47:46,349 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 04:47:46,349 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 04:47:46,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:35251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 04:47:46,605 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 04:47:47.082933 22538 mini_ranger_kms.cc:310] Created kms service
I20260808 04:47:47.276607 22538 mini_ranger_kms.cc:326] Created kudu user
I20260808 04:47:47.324045 22538 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 04:47:47 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164464, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:47.701463 22538 mini_ranger_kms.cc:384] Added ranger policy
I20260808 04:47:47.701622 22538 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzqcaaz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzqcaaz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 04:47:47.701643 22538 mini_ranger_kms.cc:240] Using host: 127.22.2.148
I20260808 04:47:47.707599 22538 mini_ranger_kms.cc:276] Ranger KMS PID: 7977
I20260808 04:47:47.707702 22538 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.2.148:42491
Aug 08, 2026 4:47:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:47:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:47:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 4:47:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 4:47:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 4:47:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 04:47:50 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786164470, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:50.507922 22538 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.751s user 0.010s sys 0.152s
I20260808 04:47:50.508064 22538 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 04:47:50.508147 22538 mini_ranger_kms.cc:401] 127.22.2.148:42491/kms/v1/keys
Aug 08 04:47:50 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164470, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
Aug 08 04:47:50 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164470, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
Aug 08 04:47:53 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:47:53.247125 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45337
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:45337
--ranger_config_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:42491/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:47:53.364364 8046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:53.364678 8046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:53.364724 8046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:53.368669 8046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:47:53.368719 8046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:53.368738 8046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:53.368754 8046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:47:53.368769 8046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:47:53.374059 8046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.2.148:42491/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:45337
--ranger_config_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:53.375011 8046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:53.375972 8046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:53.384737 8058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:53.384866 8046 server_base.cc:1034] running on GCE node
W20260808 04:47:53.384769 8053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:53.384804 8054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:53.385572 8046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:53.386688 8046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:53.387861 8046 hybrid_clock.cc:648] HybridClock initialized: now 1786164473387842 us; error 31 us; skew 500 ppm
Aug 08 04:47:53 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164473, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:53.390755 8046 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:47:53.391743 8046 webserver.cc:460] Webserver started at http://127.22.2.190:46621/ using document root <none> and password file <none>
I20260808 04:47:53.392295 8046 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:53.392388 8046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:47:53.392676 8046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 04:47:53 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164473, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:53.448511 8046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "66b111e14e2e4182bd1ec4146a72c851"
format_stamp: "Formatted at 2026-08-08 04:47:53 on dist-test-slave-04bb"
server_key: "6a840ad8d4b4f9a49310c341ef939891"
server_key_iv: "b5a33392c6b8b9642014811dface3103"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:53.449280 8046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "66b111e14e2e4182bd1ec4146a72c851"
format_stamp: "Formatted at 2026-08-08 04:47:53 on dist-test-slave-04bb"
server_key: "6a840ad8d4b4f9a49310c341ef939891"
server_key_iv: "b5a33392c6b8b9642014811dface3103"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:53.453392 8046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 04:47:53.467244 8065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:53.468767 8046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 04:47:53.468906 8046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "66b111e14e2e4182bd1ec4146a72c851"
format_stamp: "Formatted at 2026-08-08 04:47:53 on dist-test-slave-04bb"
server_key: "6a840ad8d4b4f9a49310c341ef939891"
server_key_iv: "b5a33392c6b8b9642014811dface3103"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:53.469041 8046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:53.509303 8046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:53.512461 8046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:53.512728 8046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:53.520411 8046 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:45337
I20260808 04:47:53.520426 8139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:45337 every 8 connection(s)
I20260808 04:47:53.521477 8046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:47:53.524358 8141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:47:53.525113 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 8046
I20260808 04:47:53.525246 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:47:53.525549 22538 external_mini_cluster.cc:1428] Setting key 40ae20f2fe9ed38eb93ae96bc5b9b2bb
I20260808 04:47:53.530953 8141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Bootstrap starting.
I20260808 04:47:53.533524 8141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Neither blocks nor log segments found. Creating new log.
I20260808 04:47:53.534267 8141 log.cc:824] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:47:53 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:53.536586 8141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: No bootstrap required, opened a new log
I20260808 04:47:53.539830 8141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } }
I20260808 04:47:53.540102 8141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:47:53.540176 8141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b111e14e2e4182bd1ec4146a72c851, State: Initialized, Role: FOLLOWER
I20260808 04:47:53.540706 8141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } }
I20260808 04:47:53.540835 8141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:47:53.540922 8141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:47:53.541043 8141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:47:53.541622 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:53.527022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58492 (local address 127.22.2.190:45337)
0808 04:47:53.527446 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:53.527465 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:53.527511 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:47:53.528373 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:53.528397 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:53.528829 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:53.529200 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:53.529907 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:53.531324 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:53.532391 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:53.533077 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:53.533283 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:53.536207 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:53.536213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:53.536227 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:53.536253 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:53.538530 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:53.539051 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:53.539057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:53.539062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:53.539182 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:53.539484 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:53.539487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:53.539488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:53.539885 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:53.540034 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:53.540416 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 04:47:53.540759 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":97,"threads_started":1}
I20260808 04:47:53.542022 8141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } }
I20260808 04:47:53.542382 8141 leader_election.cc:302] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66b111e14e2e4182bd1ec4146a72c851; no voters:
I20260808 04:47:53.542681 8141 leader_election.cc:288] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:47:53.542783 8150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:47:53.542999 8150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 1 LEADER]: Becoming Leader. State: Replica: 66b111e14e2e4182bd1ec4146a72c851, State: Running, Role: LEADER
I20260808 04:47:53.543344 8150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } }
I20260808 04:47:53.543970 8141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:47:53.545193 8151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66b111e14e2e4182bd1ec4146a72c851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } } }
I20260808 04:47:53.545341 8151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: This master's current role is: LEADER
I20260808 04:47:53.545658 8153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66b111e14e2e4182bd1ec4146a72c851. Latest consensus state: current_term: 1 leader_uuid: "66b111e14e2e4182bd1ec4146a72c851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } } }
I20260808 04:47:53.545744 8165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:47:53.546672 8141 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 04:47:53.549501 8153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: This master's current role is: LEADER
I20260808 04:47:53.552872 8165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:47:53.559732 8165 catalog_manager.cc:1347] Generated new cluster ID: 72ec61ed0c53442f89c60b23c0dc9ab7
I20260808 04:47:53.559792 8165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:47:53.572382 8165 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:47:53.573830 8165 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:47:53.589337 8165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Generated new TSK 0
I20260808 04:47:53.590122 8165 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:47:54.294183 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:42491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45337
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:47:54.435143 8214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:54.435416 8214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:54.435527 8214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:54.439955 8214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:54.440032 8214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:54.440114 8214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:47:54.445999 8214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.2.148:42491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:54.447105 8214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:54.448174 8214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:47:54.458042 8214 server_base.cc:1034] running on GCE node
W20260808 04:47:54.457854 8225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:54.457835 8228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:54.458348 8224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:54.458829 8214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:54.459481 8214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:54.460708 8214 hybrid_clock.cc:648] HybridClock initialized: now 1786164474460691 us; error 69 us; skew 500 ppm
Aug 08 04:47:54 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164474, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:54.464135 8214 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:47:54.465277 8214 webserver.cc:460] Webserver started at http://127.22.2.129:41393/ using document root <none> and password file <none>
I20260808 04:47:54.466025 8214 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:54.466122 8214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:47:54.466369 8214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 04:47:54 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164474, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:54.482461 8214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "669e24a21f274a65b30ca62dbfb61a5e"
format_stamp: "Formatted at 2026-08-08 04:47:54 on dist-test-slave-04bb"
server_key: "f8137d26bd0dbf4c28d277903122e161"
server_key_iv: "3515d8109829a32f85c6ceeb85bc2d91"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:54.483171 8214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "669e24a21f274a65b30ca62dbfb61a5e"
format_stamp: "Formatted at 2026-08-08 04:47:54 on dist-test-slave-04bb"
server_key: "f8137d26bd0dbf4c28d277903122e161"
server_key_iv: "3515d8109829a32f85c6ceeb85bc2d91"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:54.489993 8214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260808 04:47:54.512879 8238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:54.514674 8214 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260808 04:47:54.514823 8214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "669e24a21f274a65b30ca62dbfb61a5e"
format_stamp: "Formatted at 2026-08-08 04:47:54 on dist-test-slave-04bb"
server_key: "f8137d26bd0dbf4c28d277903122e161"
server_key_iv: "3515d8109829a32f85c6ceeb85bc2d91"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:54.514950 8214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:54.540234 8214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:54.544015 8214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:54.544265 8214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:54.545130 8214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:54.546271 8214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:54.546383 8214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:54.546515 8214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:54.546613 8214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:54.559805 8214 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:40295
I20260808 04:47:54.562423 8401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:40295 every 8 connection(s)
I20260808 04:47:54.562989 8214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:47:54.566529 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 8214
I20260808 04:47:54.566658 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:47:54.567018 22538 external_mini_cluster.cc:1428] Setting key d239570c9727956602f85dba1b08cb4b
Aug 08 04:47:54 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164474, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:54.587738 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:54.567435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:47569 (local address 127.22.2.190:45337)
0808 04:47:54.572673 (+ 5238us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:54.572685 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:54.573758 (+ 1073us) server_negotiation.cc:407] Connection header received
0808 04:47:54.575263 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:54.575267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:54.575323 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:54.575407 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:54.577010 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:54.577784 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:54.579504 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:54.579873 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:54.579946 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:54.583294 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:54.583298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:54.583301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:54.583328 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:54.585367 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:54.585977 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:54.585981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:54.585983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:54.586067 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:54.586441 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:54.586445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:54.586446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:54.586652 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:54.586745 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:54.587410 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 04:47:54.587537 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4212,"thread_start_us":119,"threads_started":1}
I20260808 04:47:54.589677 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:54.566370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45337 (local address 127.22.2.129:47569)
0808 04:47:54.572866 (+ 6496us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:54.572930 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:54.574911 (+ 1981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:54.575732 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:54.575741 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:54.576125 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:54.576846 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:54.576865 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:54.577928 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:54.577932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:54.579363 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:54.579375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:54.580151 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:54.580155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:54.580344 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:54.581070 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:54.581091 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:54.583097 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:54.585509 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:54.585515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:54.585531 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:54.585862 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:54.586168 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:54.586171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:54.586173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:54.586336 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:54.586817 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:54.586824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:54.587121 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 04:47:54.588213 (+ 1092us) client_negotiation.cc:240] Negotiation successful
0808 04:47:54.588585 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4980,"thread_start_us":145,"threads_started":1}
I20260808 04:47:54.591506 8402 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45337
I20260808 04:47:54.591773 8402 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:54.592392 8402 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:47:54.594420 8086 ts_manager.cc:194] Registered new tserver with Master: 669e24a21f274a65b30ca62dbfb61a5e (127.22.2.129:40295)
I20260808 04:47:54.595955 8086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:47569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:47:54.620154 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:42491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45337
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:47:54 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786164474, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 04:47:54.791090 8410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:54.791433 8410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:54.791517 8410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:54.797587 8410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:54.797684 8410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:54.797788 8410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:47:54.805536 8410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.2.148:42491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:54.807296 8410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:54.808835 8410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:54.820972 8419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:54.821348 8421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:54.822316 8417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:54.822606 8410 server_base.cc:1034] running on GCE node
I20260808 04:47:54.823094 8410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:54.823972 8410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:54.825204 8410 hybrid_clock.cc:648] HybridClock initialized: now 1786164474825121 us; error 102 us; skew 500 ppm
Aug 08 04:47:54 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164474, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:54.829306 8410 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:47:54.830507 8410 webserver.cc:460] Webserver started at http://127.22.2.130:40475/ using document root <none> and password file <none>
I20260808 04:47:54.831367 8410 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:54.831528 8410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:47:54.831897 8410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 04:47:54 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164474, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:54.862283 8410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "eb0f58776b6b44a583e6097d2c8b1e3f"
format_stamp: "Formatted at 2026-08-08 04:47:54 on dist-test-slave-04bb"
server_key: "582b9701c49c9bd304c42e7169fb09ea"
server_key_iv: "b1809030980404665fe27f8ee5ef048e"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:54.866248 8410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "eb0f58776b6b44a583e6097d2c8b1e3f"
format_stamp: "Formatted at 2026-08-08 04:47:54 on dist-test-slave-04bb"
server_key: "582b9701c49c9bd304c42e7169fb09ea"
server_key_iv: "b1809030980404665fe27f8ee5ef048e"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:54.872785 8410 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 04:47:54.900624 8429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:54.902482 8410 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 04:47:54.902632 8410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "eb0f58776b6b44a583e6097d2c8b1e3f"
format_stamp: "Formatted at 2026-08-08 04:47:54 on dist-test-slave-04bb"
server_key: "582b9701c49c9bd304c42e7169fb09ea"
server_key_iv: "b1809030980404665fe27f8ee5ef048e"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:54.902770 8410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:54.924322 8410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:54.928244 8410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:54.928473 8410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:54.929273 8410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:54.930454 8410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:54.930527 8410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:54.930601 8410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:54.930630 8410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:54.945472 8410 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:36901
I20260808 04:47:54.945665 8591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:36901 every 8 connection(s)
I20260808 04:47:54.946723 8410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:47:54.951185 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 8410
I20260808 04:47:54.951351 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:47:54.951735 22538 external_mini_cluster.cc:1428] Setting key 7201bd2beeb6b1f92eee045b43d123c0
Aug 08 04:47:54 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164474, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:54.966627 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:54.948959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:37923 (local address 127.22.2.190:45337)
0808 04:47:54.949104 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:54.949109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:54.950129 (+ 1020us) server_negotiation.cc:407] Connection header received
0808 04:47:54.951390 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:54.951397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:54.951467 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:54.951534 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:54.953888 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:54.954819 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:54.956699 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:54.957145 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:54.957258 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:54.960921 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:54.960928 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:54.960934 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:54.960975 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:54.963198 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:54.963993 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:54.963998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:54.964002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:54.964108 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:54.964966 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:54.964972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:54.964977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:54.965275 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:54.965377 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:54.966209 (+ 832us) server_negotiation.cc:299] Negotiation successful
0808 04:47:54.966409 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:47:54.968627 8597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:54.949450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45337 (local address 127.22.2.130:37923)
0808 04:47:54.949956 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:54.950015 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:54.951076 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:54.952002 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:54.952011 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:54.952425 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:54.953696 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:54.953718 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:54.955010 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:54.955016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:54.956498 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:54.956508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:54.957508 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:54.957515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:54.957757 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:54.958543 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:54.958571 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:54.960686 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:54.963380 (+ 2694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:54.963389 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:54.963410 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:54.963849 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:54.964252 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:54.964258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:54.964262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:54.964472 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:54.965463 (+ 991us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:54.965472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:54.965852 (+ 380us) client_negotiation.cc:769] Sending connection context
0808 04:47:54.967165 (+ 1313us) client_negotiation.cc:240] Negotiation successful
0808 04:47:54.967532 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:47:54.970304 8592 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45337
I20260808 04:47:54.970579 8592 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:54.971233 8592 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
I20260808 04:47:54.972820 8086 ts_manager.cc:194] Registered new tserver with Master: eb0f58776b6b44a583e6097d2c8b1e3f (127.22.2.130:36901)
I20260808 04:47:54.973652 8086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:37923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:47:54.999375 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:42491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45337
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:47:55 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164474, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
W20260808 04:47:55.192145 8600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:55.192487 8600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:55.192576 8600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:55.198469 8600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:55.198539 8600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:55.198623 8600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:47:55.207665 8600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.2.148:42491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:55.209348 8600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:55.210923 8600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:55.225997 8609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:55.225978 8608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:55.226274 8613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:55.226992 8600 server_base.cc:1034] running on GCE node
I20260808 04:47:55.227566 8600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:55.228467 8600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:55.229702 8600 hybrid_clock.cc:648] HybridClock initialized: now 1786164475229685 us; error 50 us; skew 500 ppm
Aug 08 04:47:55 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164475, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:55.234377 8600 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:47:55.235610 8600 webserver.cc:460] Webserver started at http://127.22.2.131:42365/ using document root <none> and password file <none>
I20260808 04:47:55.236388 8600 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:55.236481 8600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:47:55.236784 8600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 04:47:55 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164475, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:55.272152 8600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "19ee6cb573b748cca3e819b596407728"
format_stamp: "Formatted at 2026-08-08 04:47:55 on dist-test-slave-04bb"
server_key: "af1ded4a919a48e7fc4465fdbc549579"
server_key_iv: "ebda23dbb6a628fab16b06cfa2dbc0ce"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:55.273025 8600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "19ee6cb573b748cca3e819b596407728"
format_stamp: "Formatted at 2026-08-08 04:47:55 on dist-test-slave-04bb"
server_key: "af1ded4a919a48e7fc4465fdbc549579"
server_key_iv: "ebda23dbb6a628fab16b06cfa2dbc0ce"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:55.279090 8600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 04:47:55.302192 8623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:55.304298 8600 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 04:47:55.304569 8600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "19ee6cb573b748cca3e819b596407728"
format_stamp: "Formatted at 2026-08-08 04:47:55 on dist-test-slave-04bb"
server_key: "af1ded4a919a48e7fc4465fdbc549579"
server_key_iv: "ebda23dbb6a628fab16b06cfa2dbc0ce"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:55.304834 8600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:55.330238 8600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:55.334525 8600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:55.334822 8600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:55.335783 8600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:55.337095 8600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:55.337222 8600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:55.337343 8600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:55.337431 8600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:55.360260 8600 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:37707
I20260808 04:47:55.361742 8600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:47:55.363299 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 8600
I20260808 04:47:55.363440 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:47:55.363771 22538 external_mini_cluster.cc:1428] Setting key 8537c760bbb062cdd66e4fd7967ebf53
I20260808 04:47:55.364254 8789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:37707 every 8 connection(s)
Aug 08 04:47:55 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164475, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:55.382802 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:55.366513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:42443 (local address 127.22.2.190:45337)
0808 04:47:55.366647 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:55.366651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:55.366921 (+ 270us) server_negotiation.cc:407] Connection header received
0808 04:47:55.368395 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:55.368400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:55.368465 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:55.368573 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:55.370439 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:55.371354 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:55.373454 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:55.373845 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:55.373955 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:55.377900 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:55.377905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:55.377908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:55.377943 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:55.380036 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:55.380773 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:55.380777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:55.380779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:55.380885 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:55.381343 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:55.381347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:55.381350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:55.381602 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:55.381718 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:55.382474 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 04:47:55.382616 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:47:55.383999 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:55.365308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45337 (local address 127.22.2.131:42443)
0808 04:47:55.366744 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:55.366808 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:55.368108 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:55.369061 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:55.369088 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:55.369505 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:55.370235 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:55.370256 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:55.371554 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:55.371558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:55.373243 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:55.373257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:55.374398 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:55.374404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:55.374792 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:55.375660 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:55.375684 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:55.377681 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:55.380168 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:55.380175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:55.380192 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:55.380627 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:55.381005 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:55.381008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:55.381011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:55.381199 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:55.381771 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:55.381788 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:55.382142 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 04:47:55.382363 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 04:47:55.382766 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":166,"threads_started":1}
I20260808 04:47:55.385874 8790 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45337
I20260808 04:47:55.386152 8790 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:55.386771 8790 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
I20260808 04:47:55.388025 8086 ts_manager.cc:194] Registered new tserver with Master: 19ee6cb573b748cca3e819b596407728 (127.22.2.131:37707)
I20260808 04:47:55.388861 8086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:42443
I20260808 04:47:55.391214 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:47:55.392041 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 8214
I20260808 04:47:55.419731 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 8410
I20260808 04:47:55.432806 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 8600
I20260808 04:47:55.446360 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 8046
I20260808 04:47:55.455431 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45337
--webserver_interface=127.22.2.190
--webserver_port=46621
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:45337
--ranger_config_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:42491/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:42491/kms with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:47:55.570235 8794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:55.570493 8794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:55.570533 8794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:55.574602 8794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:47:55.574659 8794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:55.574679 8794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:55.574695 8794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:47:55.574712 8794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:47:55.580008 8794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:42491/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:45337
--ranger_config_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=46621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:55.581022 8794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:55.581976 8794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:55.590762 8805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:55.590818 8802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:55.590757 8799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:55.591039 8794 server_base.cc:1034] running on GCE node
I20260808 04:47:55.591687 8794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:55.592847 8794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:55.594039 8794 hybrid_clock.cc:648] HybridClock initialized: now 1786164475594020 us; error 32 us; skew 500 ppm
Aug 08 04:47:55 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164475, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:55.596841 8794 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:47:55.597764 8794 webserver.cc:460] Webserver started at http://127.22.2.190:46621/ using document root <none> and password file <none>
I20260808 04:47:55.598304 8794 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:55.598402 8794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 04:47:55 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164475, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:55.646422 8794 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:47:55.648818 8819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:55.650205 8794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:47:55.650363 8794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "66b111e14e2e4182bd1ec4146a72c851"
format_stamp: "Formatted at 2026-08-08 04:47:53 on dist-test-slave-04bb"
server_key: "6a840ad8d4b4f9a49310c341ef939891"
server_key_iv: "b5a33392c6b8b9642014811dface3103"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:55.650938 8794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:55.674743 8794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:55.677877 8794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:55.678097 8794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:55.685613 8794 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:45337
I20260808 04:47:55.685624 8897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:45337 every 8 connection(s)
I20260808 04:47:55.686679 8794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:47:55.690963 8900 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:47:55.692612 8900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Bootstrap starting.
I20260808 04:47:55.693419 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 8794
I20260808 04:47:55.693931 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:42491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:40295
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:45337
--webserver_port=41393
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:42491/kms with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:47:55.710000 8900 log.cc:824] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Log is configured to *not* fsync() on all Append() calls
I20260808 04:47:55.716703 8900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:47:55.717108 8900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Bootstrap complete.
I20260808 04:47:55.721452 8900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } }
I20260808 04:47:55.721835 8900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b111e14e2e4182bd1ec4146a72c851, State: Initialized, Role: FOLLOWER
I20260808 04:47:55.722390 8900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } }
I20260808 04:47:55.722512 8900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:47:55.722597 8900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:47:55.722714 8900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:47:55.724598 8900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } }
I20260808 04:47:55.724936 8900 leader_election.cc:302] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66b111e14e2e4182bd1ec4146a72c851; no voters:
I20260808 04:47:55.725270 8900 leader_election.cc:288] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 04:47:55.725365 8908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 04:47:55.725558 8908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [term 2 LEADER]: Becoming Leader. State: Replica: 66b111e14e2e4182bd1ec4146a72c851, State: Running, Role: LEADER
I20260808 04:47:55.725838 8908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } }
I20260808 04:47:55.726538 8900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:47:55.726788 8908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66b111e14e2e4182bd1ec4146a72c851. Latest consensus state: current_term: 2 leader_uuid: "66b111e14e2e4182bd1ec4146a72c851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } } }
I20260808 04:47:55.726910 8908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: This master's current role is: LEADER
I20260808 04:47:55.726864 8909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "66b111e14e2e4182bd1ec4146a72c851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66b111e14e2e4182bd1ec4146a72c851" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45337 } } }
I20260808 04:47:55.727221 8909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851 [sys.catalog]: This master's current role is: LEADER
I20260808 04:47:55.727522 8913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:47:55.730934 8913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:47:55.731422 8900 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 04:47:55.731647 8913 catalog_manager.cc:1259] Loaded cluster ID: 72ec61ed0c53442f89c60b23c0dc9ab7
I20260808 04:47:55.731704 8913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:47:55.737097 8913 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:47:55.738317 8913 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Loaded TSK: 0
I20260808 04:47:55.739487 8913 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 04:47:55.840193 8905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:55.840546 8905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:55.840620 8905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:55.846450 8905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:55.846524 8905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:55.846607 8905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:47:55.854159 8905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:42491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:40295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=41393
--tserver_master_addrs=127.22.2.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:55.855500 8905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:55.856884 8905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:55.869904 8953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:55.870289 8959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:55.870575 8955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:55.871418 8905 server_base.cc:1034] running on GCE node
I20260808 04:47:55.871971 8905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:55.872787 8905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:55.873957 8905 hybrid_clock.cc:648] HybridClock initialized: now 1786164475873885 us; error 88 us; skew 500 ppm
Aug 08 04:47:55 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164475, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:55.877777 8905 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:47:55.878934 8905 webserver.cc:460] Webserver started at http://127.22.2.129:41393/ using document root <none> and password file <none>
I20260808 04:47:55.879668 8905 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:55.879778 8905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 04:47:55 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164475, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:55.907891 8905 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.000s
I20260808 04:47:55.926185 8971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:55.928155 8905 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.002s sys 0.000s
I20260808 04:47:55.928310 8905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "669e24a21f274a65b30ca62dbfb61a5e"
format_stamp: "Formatted at 2026-08-08 04:47:54 on dist-test-slave-04bb"
server_key: "f8137d26bd0dbf4c28d277903122e161"
server_key_iv: "3515d8109829a32f85c6ceeb85bc2d91"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:55.929133 8905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:55.975365 8905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:55.978674 8905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:55.978907 8905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:55.979550 8905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:55.980580 8905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:55.980656 8905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:55.980721 8905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:55.980748 8905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:55.993151 8905 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:40295
I20260808 04:47:55.994369 8905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:47:55.995240 9132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:40295 every 8 connection(s)
I20260808 04:47:56.003253 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 8905
I20260808 04:47:56.003815 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:42491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:36901
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:45337
--webserver_port=40475
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:42491/kms with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:47:56 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164475, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:56.017618 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:55.997626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45337 (local address 127.22.2.129:45443)
0808 04:47:55.998948 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:55.999003 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:56.000176 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:56.002358 (+ 2182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:56.002367 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:56.002735 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:56.003407 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:56.003428 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.005415 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.005422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:56.006930 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:56.006942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.008286 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.008292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:56.008518 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:56.009397 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:56.009422 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:56.011016 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:56.013297 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:56.013307 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:56.013323 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:56.013590 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:56.014697 (+ 1107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:56.014700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:56.014702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:56.014851 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:56.015595 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:56.015601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:56.015913 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 04:47:56.016125 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 04:47:56.016484 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"thread_start_us":164,"threads_started":1}
I20260808 04:47:56.019550 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:55.997547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:45443 (local address 127.22.2.190:45337)
0808 04:47:55.998161 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:55.998201 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:56.000540 (+ 2339us) server_negotiation.cc:407] Connection header received
0808 04:47:56.000880 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:56.000894 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:56.001296 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:56.001662 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:56.003620 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.005248 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.007076 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.007857 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.008025 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:56.011224 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:56.011233 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:56.011252 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:56.011278 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:56.013163 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:56.014434 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:56.014440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:56.014444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:56.014564 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:56.014957 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:56.014961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:56.014963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:56.015395 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:56.015520 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:56.018307 (+ 2787us) server_negotiation.cc:299] Negotiation successful
0808 04:47:56.018609 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":111,"threads_started":1}
I20260808 04:47:56.022660 9133 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45337
I20260808 04:47:56.022934 9133 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:56.023538 9133 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
W20260808 04:47:56.025852 9133 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:45337 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 04:47:56.025982 9133 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:56.026255 9133 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
I20260808 04:47:56.027091 9133 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:56.027280 9133 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
W20260808 04:47:56.028153 9133 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:47:56.178139 9142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:56.178508 9142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:56.178602 9142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:56.184741 9142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:56.184823 9142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:56.184911 9142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:47:56.192958 9142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:42491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:36901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=40475
--tserver_master_addrs=127.22.2.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:56.194344 9142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:56.195732 9142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:56.208791 9149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:56.208791 9148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:56.208748 9151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:56.209736 9142 server_base.cc:1034] running on GCE node
I20260808 04:47:56.210326 9142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:56.211139 9142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:56.212637 9142 hybrid_clock.cc:648] HybridClock initialized: now 1786164476212608 us; error 45 us; skew 500 ppm
Aug 08 04:47:56 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164476, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:56.216650 9142 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:47:56.217883 9142 webserver.cc:460] Webserver started at http://127.22.2.130:40475/ using document root <none> and password file <none>
I20260808 04:47:56.218708 9142 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:56.218827 9142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 04:47:56 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164476, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:56.246325 9142 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:47:56.249261 9168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:56.251103 9142 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 04:47:56.251238 9142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "eb0f58776b6b44a583e6097d2c8b1e3f"
format_stamp: "Formatted at 2026-08-08 04:47:54 on dist-test-slave-04bb"
server_key: "582b9701c49c9bd304c42e7169fb09ea"
server_key_iv: "b1809030980404665fe27f8ee5ef048e"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:56.251979 9142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:56.273859 9142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:56.277267 9142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:56.277468 9142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:56.278160 9142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:56.279098 9142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:56.279170 9142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:56.279243 9142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:56.279297 9142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:56.290802 9142 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:36901
I20260808 04:47:56.292264 9142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:47:56.293203 9330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:36901 every 8 connection(s)
I20260808 04:47:56.301265 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 9142
I20260808 04:47:56.301818 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:42491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:37707
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:45337
--webserver_port=42365
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:42491/kms with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:47:56 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164476, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:56.314750 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:56.300200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:56123 (local address 127.22.2.190:45337)
0808 04:47:56.300354 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:56.300358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:56.301907 (+ 1549us) server_negotiation.cc:407] Connection header received
0808 04:47:56.303361 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:56.303365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:56.303433 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:56.303539 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:56.305215 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.306043 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.307797 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.308043 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.308173 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:56.310688 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:56.310694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:56.310698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:56.310726 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:56.312301 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:56.312993 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:56.312997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:56.312998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:56.313077 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:56.313448 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:56.313451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:56.313455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:56.313672 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:56.313787 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:56.314437 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 04:47:56.314576 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 04:47:56.316612 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:56.300749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45337 (local address 127.22.2.130:56123)
0808 04:47:56.301710 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:56.301771 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:56.303042 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:56.303886 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:56.303896 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:56.304263 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:56.305006 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:56.305029 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.306213 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.306217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:56.307650 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:56.307664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.308216 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.308220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:56.308402 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:56.309130 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:56.309147 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:56.310504 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:56.312430 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:56.312437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:56.312453 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:56.312848 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:56.313178 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:56.313182 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:56.313184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:56.313339 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:56.313785 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:56.313792 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:56.314110 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 04:47:56.315212 (+ 1102us) client_negotiation.cc:240] Negotiation successful
0808 04:47:56.315561 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":137,"threads_started":1}
I20260808 04:47:56.318233 9332 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45337
I20260808 04:47:56.318504 9332 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:56.319118 9332 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
W20260808 04:47:56.320899 9332 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:45337 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 04:47:56.321034 9332 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:56.321201 9332 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
I20260808 04:47:56.322134 9332 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:56.322294 9332 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
W20260808 04:47:56.323093 9332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:47:56.476585 9336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:47:56.476955 9336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:47:56.477109 9336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:47:56.483188 9336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:47:56.483306 9336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:47:56.483424 9336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:47:56.491350 9336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:42491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:37707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=42365
--tserver_master_addrs=127.22.2.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:47:56.492883 9336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:47:56.494329 9336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:47:56.505854 9342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:56.505957 9345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:47:56.505841 9347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:47:56.506234 9336 server_base.cc:1034] running on GCE node
I20260808 04:47:56.506675 9336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:47:56.507433 9336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:47:56.508618 9336 hybrid_clock.cc:648] HybridClock initialized: now 1786164476508580 us; error 45 us; skew 500 ppm
Aug 08 04:47:56 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164476, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:56.512137 9336 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:47:56.513346 9336 webserver.cc:460] Webserver started at http://127.22.2.131:42365/ using document root <none> and password file <none>
I20260808 04:47:56.514111 9336 fs_manager.cc:359] Metadata directory not provided
I20260808 04:47:56.514223 9336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 04:47:56 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164476, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:56.538384 9336 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.000s
I20260808 04:47:56.541656 9372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:56.543443 9336 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:47:56.543596 9336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "19ee6cb573b748cca3e819b596407728"
format_stamp: "Formatted at 2026-08-08 04:47:55 on dist-test-slave-04bb"
server_key: "af1ded4a919a48e7fc4465fdbc549579"
server_key_iv: "ebda23dbb6a628fab16b06cfa2dbc0ce"
server_key_version: "kuduclusterkey@0"
I20260808 04:47:56.544359 9336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:47:56.561034 9336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:47:56.564637 9336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:47:56.564843 9336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:47:56.565578 9336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:47:56.566690 9336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:47:56.566766 9336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:56.566823 9336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:47:56.566859 9336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:47:56.578348 9336 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:37707
I20260808 04:47:56.578423 9540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:37707 every 8 connection(s)
I20260808 04:47:56.579748 9336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:47:56.587678 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 9336
Aug 08 04:47:56 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164476, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:47:56.597177 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:56.582671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:54735 (local address 127.22.2.190:45337)
0808 04:47:56.582819 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:56.582824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:56.583914 (+ 1090us) server_negotiation.cc:407] Connection header received
0808 04:47:56.585286 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:56.585290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:56.585351 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:56.585449 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:56.586890 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.587748 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.589516 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.589876 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.589958 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:56.592948 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:56.592954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:56.592961 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:56.592985 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:56.594596 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:56.595231 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:56.595235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:56.595236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:56.595325 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:56.595690 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:56.595694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:56.595696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:56.595873 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:56.595983 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:56.596568 (+ 585us) server_negotiation.cc:299] Negotiation successful
0808 04:47:56.596998 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":62}
I20260808 04:47:56.597950 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:56.583135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45337 (local address 127.22.2.131:54735)
0808 04:47:56.583736 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:56.583797 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:56.585027 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:56.585670 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:56.585680 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:56.586033 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:47:56.586710 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:56.586728 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.587937 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.587941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:56.589364 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:56.589376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.590091 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.590095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:56.590285 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:56.590974 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:47:56.590995 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:47:56.592768 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:47:56.594717 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:56.594722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:56.594737 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:56.595110 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:56.595428 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:47:56.595431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:47:56.595433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:47:56.595583 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:47:56.595981 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:47:56.595989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:47:56.596242 (+ 253us) client_negotiation.cc:769] Sending connection context
0808 04:47:56.596449 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 04:47:56.596809 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":150,"threads_started":1}
I20260808 04:47:56.599244 9541 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45337
I20260808 04:47:56.599521 9541 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:56.600103 9541 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
I20260808 04:47:56.601430 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:56.588985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37228 (local address 127.22.2.190:45337)
0808 04:47:56.589716 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:56.589724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:56.589739 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:47:56.589776 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:56.589779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:56.589822 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:56.589909 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:56.591726 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.592335 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.593634 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:56.593951 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:56.594078 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:56.596384 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:56.596390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:56.596396 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:56.596428 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:56.598152 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:56.598691 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:56.598698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:56.598700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:56.598792 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:56.599142 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:56.599146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:56.599147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:56.599361 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:56.599569 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:56.600979 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0808 04:47:56.601109 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":651,"thread_start_us":109,"threads_started":1}
I20260808 04:47:56.601609 8844 ts_manager.cc:194] Registered new tserver with Master: 19ee6cb573b748cca3e819b596407728 (127.22.2.131:37707)
I20260808 04:47:56.602718 8844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:54735
Aug 08 04:47:56 dist-test-slave-04bb krb5kdc[7406](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786164476, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:47:57.028388 9133 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:57.028761 9133 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
I20260808 04:47:57.029634 8844 ts_manager.cc:194] Registered new tserver with Master: 669e24a21f274a65b30ca62dbfb61a5e (127.22.2.129:40295)
I20260808 04:47:57.030467 8844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:45443
Aug 08 04:47:57 dist-test-slave-04bb krb5kdc[7406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164476, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 04:47:57.323336 9332 heartbeater.cc:464] Registering TS with master...
I20260808 04:47:57.323652 9332 heartbeater.cc:511] Master 127.22.2.190:45337 requested a full tablet report, sending...
I20260808 04:47:57.324491 8844 ts_manager.cc:194] Registered new tserver with Master: eb0f58776b6b44a583e6097d2c8b1e3f (127.22.2.130:36901)
I20260808 04:47:57.325277 8844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:56123
I20260808 04:47:57.327652 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:47:57.343968 9566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.329386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.22.2.190:45337)
0808 04:47:57.331960 (+ 2574us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:57.331964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:57.331977 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:47:57.332014 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:57.332017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:57.332060 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:57.332144 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:47:57.333054 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.333868 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.335164 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.335529 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.335629 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.340169 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:47:57.340173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:47:57.340176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:47:57.340204 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:47:57.342107 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:57.342611 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:57.342615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:57.342617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:57.342709 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:47:57.343023 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:47:57.343026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:47:57.343028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:47:57.343232 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:47:57.343361 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:57.343659 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 04:47:57.343795 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2499,"thread_start_us":74,"threads_started":1}
I20260808 04:47:57.347975 8844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:47:57.352743 8844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:47:57.372151 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.364787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.22.2.129:40295)
0808 04:47:57.365053 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:57.365061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:57.366529 (+ 1468us) server_negotiation.cc:407] Connection header received
0808 04:47:57.366627 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:57.366633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:57.367029 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:57.367195 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:57.367489 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.368652 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.370347 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.371182 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.371266 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.371433 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:57.371886 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 04:47:57.371995 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
I20260808 04:47:57.372645 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.363964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:40295 (local address 127.0.0.1:56066)
0808 04:47:57.366417 (+ 2453us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:57.366429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:57.366513 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:57.367197 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:57.367204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:57.367229 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:57.367360 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.367366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.368806 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.368810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.370183 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.370195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.371547 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.371550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.371659 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.371701 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 04:47:57.372344 (+ 643us) client_negotiation.cc:240] Negotiation successful
0808 04:47:57.372482 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2394,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260808 04:47:57.376191 9034 tablet_service.cc:1467] Processing CreateTablet for tablet b1eed400c19c46fb81f254e4e9f8f6cd (DEFAULT_TABLE table=test-table [id=fb1ff534d2a9485994ef8521dae8409b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:47:57.377511 9034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1eed400c19c46fb81f254e4e9f8f6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:47:57.378153 9579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.365177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:36901 (local address 127.0.0.1:53990)
0808 04:47:57.368749 (+ 3572us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:57.368762 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:57.368860 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:57.369494 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:57.369497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:57.369514 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:57.369593 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.369598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.371766 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.371769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.373666 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.373678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.377749 (+ 4071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.377753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.377857 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.377881 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 04:47:57.377950 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 04:47:57.378017 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3508,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260808 04:47:57.378858 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.365333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53990 (local address 127.22.2.130:36901)
0808 04:47:57.365693 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:57.365702 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:57.368955 (+ 3253us) server_negotiation.cc:407] Connection header received
0808 04:47:57.369035 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:57.369040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:57.369231 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:57.369363 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:57.369711 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.371635 (+ 1924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.375871 (+ 4236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.377445 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.377551 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.377686 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:57.378606 (+ 920us) server_negotiation.cc:299] Negotiation successful
0808 04:47:57.378709 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":95,"threads_started":1}
I20260808 04:47:57.385990 9237 tablet_service.cc:1467] Processing CreateTablet for tablet b1eed400c19c46fb81f254e4e9f8f6cd (DEFAULT_TABLE table=test-table [id=fb1ff534d2a9485994ef8521dae8409b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:47:57.387053 9237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1eed400c19c46fb81f254e4e9f8f6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:47:57.389353 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.363917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:37707 (local address 127.0.0.1:48798)
0808 04:47:57.365705 (+ 1788us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:57.365928 (+ 223us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:57.366123 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:57.374235 (+ 8112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:57.374238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:57.374256 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:57.374352 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.374357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.379111 (+ 4754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.379114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.381203 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.381214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.388831 (+ 7617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.388838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.388970 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.389007 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 04:47:57.389068 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 04:47:57.389175 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1623,"thread_start_us":100,"threads_started":1}
I20260808 04:47:57.390551 9578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.365129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.22.2.131:37707)
0808 04:47:57.373796 (+ 8667us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:57.373806 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:57.373829 (+ 23us) server_negotiation.cc:407] Connection header received
0808 04:47:57.373907 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:57.373912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:57.374095 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:57.374497 (+ 402us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:57.374629 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.375732 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.381369 (+ 5637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.382211 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.389689 (+ 7478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.390192 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:57.390260 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 04:47:57.390379 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8001,"thread_start_us":90,"threads_started":1}
I20260808 04:47:57.393380 9439 tablet_service.cc:1467] Processing CreateTablet for tablet b1eed400c19c46fb81f254e4e9f8f6cd (DEFAULT_TABLE table=test-table [id=fb1ff534d2a9485994ef8521dae8409b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:47:57.394542 9439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1eed400c19c46fb81f254e4e9f8f6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:47:57.396893 9581 tablet_bootstrap.cc:492] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e: Bootstrap starting.
I20260808 04:47:57.398115 9583 tablet_bootstrap.cc:492] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: Bootstrap starting.
I20260808 04:47:57.399345 9581 tablet_bootstrap.cc:654] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e: Neither blocks nor log segments found. Creating new log.
I20260808 04:47:57.400125 9583 tablet_bootstrap.cc:654] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: Neither blocks nor log segments found. Creating new log.
I20260808 04:47:57.400199 9581 log.cc:824] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e: Log is configured to *not* fsync() on all Append() calls
I20260808 04:47:57.401130 9583 log.cc:824] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: Log is configured to *not* fsync() on all Append() calls
I20260808 04:47:57.403502 9583 tablet_bootstrap.cc:492] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: No bootstrap required, opened a new log
I20260808 04:47:57.403693 9583 ts_tablet_manager.cc:1397] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 04:47:57.408140 9586 tablet_bootstrap.cc:492] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: Bootstrap starting.
I20260808 04:47:57.408190 9583 raft_consensus.cc:348] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.408582 9583 raft_consensus.cc:374] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:47:57.408689 9583 raft_consensus.cc:729] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f58776b6b44a583e6097d2c8b1e3f, State: Initialized, Role: FOLLOWER
I20260808 04:47:57.409267 9583 consensus_queue.cc:260] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.409770 9581 tablet_bootstrap.cc:492] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e: No bootstrap required, opened a new log
I20260808 04:47:57.410017 9581 ts_tablet_manager.cc:1397] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.006s
I20260808 04:47:57.410441 9586 tablet_bootstrap.cc:654] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: Neither blocks nor log segments found. Creating new log.
I20260808 04:47:57.410567 9583 ts_tablet_manager.cc:1428] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 04:47:57.410983 9332 heartbeater.cc:503] Master 127.22.2.190:45337 was elected leader, sending a full tablet report...
I20260808 04:47:57.411342 9586 log.cc:824] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: Log is configured to *not* fsync() on all Append() calls
I20260808 04:47:57.413966 9581 raft_consensus.cc:348] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.414240 9581 raft_consensus.cc:374] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:47:57.414333 9581 raft_consensus.cc:729] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669e24a21f274a65b30ca62dbfb61a5e, State: Initialized, Role: FOLLOWER
I20260808 04:47:57.414852 9581 consensus_queue.cc:260] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.416015 9581 ts_tablet_manager.cc:1428] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260808 04:47:57.416424 9133 heartbeater.cc:503] Master 127.22.2.190:45337 was elected leader, sending a full tablet report...
I20260808 04:47:57.418735 9586 tablet_bootstrap.cc:492] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: No bootstrap required, opened a new log
I20260808 04:47:57.418948 9586 ts_tablet_manager.cc:1397] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260808 04:47:57.422708 9586 raft_consensus.cc:348] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.422955 9586 raft_consensus.cc:374] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:47:57.423043 9586 raft_consensus.cc:729] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19ee6cb573b748cca3e819b596407728, State: Initialized, Role: FOLLOWER
I20260808 04:47:57.423501 9586 consensus_queue.cc:260] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.424142 9541 heartbeater.cc:503] Master 127.22.2.190:45337 was elected leader, sending a full tablet report...
I20260808 04:47:57.424517 9586 ts_tablet_manager.cc:1428] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:47:57.456833 9591 raft_consensus.cc:482] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:47:57.457083 9591 raft_consensus.cc:504] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.458374 9591 leader_election.cc:288] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19ee6cb573b748cca3e819b596407728 (127.22.2.131:37707), eb0f58776b6b44a583e6097d2c8b1e3f (127.22.2.130:36901)
I20260808 04:47:57.464210 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.459009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:41971 (local address 127.22.2.130:36901)
0808 04:47:57.459137 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:57.459142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:57.459770 (+ 628us) server_negotiation.cc:407] Connection header received
0808 04:47:57.459966 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:57.459972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:57.460031 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:57.460081 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:57.460466 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.461201 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.462863 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.463459 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.463555 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.463608 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:57.463986 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 04:47:57.464083 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 04:47:57.464689 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.458664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:36901 (local address 127.22.2.129:41971)
0808 04:47:57.459680 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:57.459694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:57.459833 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:57.460220 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:57.460223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:57.460239 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:57.460339 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.460345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.461322 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.461325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.462715 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.462732 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.463715 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.463718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.463835 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.463862 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 04:47:57.464407 (+ 545us) client_negotiation.cc:240] Negotiation successful
0808 04:47:57.464495 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":98,"thread_start_us":99,"threads_started":1}
I20260808 04:47:57.465373 9578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.459068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:41631 (local address 127.22.2.131:37707)
0808 04:47:57.459184 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:57.459188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:57.459402 (+ 214us) server_negotiation.cc:407] Connection header received
0808 04:47:57.459567 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:57.459572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:57.459625 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:57.459696 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:47:57.460140 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.460969 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.462863 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.463473 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.463555 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.463608 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:57.465172 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0808 04:47:57.465269 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 04:47:57.465582 9266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1eed400c19c46fb81f254e4e9f8f6cd" candidate_uuid: "669e24a21f274a65b30ca62dbfb61a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" is_pre_election: true
I20260808 04:47:57.465804 9594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.458565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:37707 (local address 127.22.2.129:41631)
0808 04:47:57.459309 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:57.459321 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:47:57.459449 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:47:57.459828 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:47:57.459831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:47:57.459844 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:47:57.460012 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.460020 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.461103 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.461106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.462715 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:47:57.462727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.464908 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.464914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:47:57.465030 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.465056 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 04:47:57.465577 (+ 521us) client_negotiation.cc:240] Negotiation successful
0808 04:47:57.465656 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":58,"thread_start_us":128,"threads_started":1}
I20260808 04:47:57.465924 9266 raft_consensus.cc:2454] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 669e24a21f274a65b30ca62dbfb61a5e in term 0.
I20260808 04:47:57.466410 8975 leader_election.cc:302] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 669e24a21f274a65b30ca62dbfb61a5e, eb0f58776b6b44a583e6097d2c8b1e3f; no voters:
I20260808 04:47:57.466437 9471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1eed400c19c46fb81f254e4e9f8f6cd" candidate_uuid: "669e24a21f274a65b30ca62dbfb61a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19ee6cb573b748cca3e819b596407728" is_pre_election: true
I20260808 04:47:57.466697 9591 raft_consensus.cc:2781] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:47:57.466780 9591 raft_consensus.cc:482] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:47:57.466766 9471 raft_consensus.cc:2454] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 669e24a21f274a65b30ca62dbfb61a5e in term 0.
I20260808 04:47:57.466821 9591 raft_consensus.cc:3037] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:47:57.467999 9591 raft_consensus.cc:504] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.468339 9591 leader_election.cc:288] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [CANDIDATE]: Term 1 election: Requested vote from peers 19ee6cb573b748cca3e819b596407728 (127.22.2.131:37707), eb0f58776b6b44a583e6097d2c8b1e3f (127.22.2.130:36901)
I20260808 04:47:57.468816 9471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1eed400c19c46fb81f254e4e9f8f6cd" candidate_uuid: "669e24a21f274a65b30ca62dbfb61a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19ee6cb573b748cca3e819b596407728"
I20260808 04:47:57.468897 9266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1eed400c19c46fb81f254e4e9f8f6cd" candidate_uuid: "669e24a21f274a65b30ca62dbfb61a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f"
I20260808 04:47:57.468933 9471 raft_consensus.cc:3037] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:47:57.469007 9266 raft_consensus.cc:3037] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:47:57.470203 9471 raft_consensus.cc:2454] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 669e24a21f274a65b30ca62dbfb61a5e in term 1.
I20260808 04:47:57.470203 9266 raft_consensus.cc:2454] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 669e24a21f274a65b30ca62dbfb61a5e in term 1.
I20260808 04:47:57.470507 8975 leader_election.cc:302] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 669e24a21f274a65b30ca62dbfb61a5e, eb0f58776b6b44a583e6097d2c8b1e3f; no voters:
I20260808 04:47:57.470760 9591 raft_consensus.cc:2781] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:47:57.471027 9591 raft_consensus.cc:686] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [term 1 LEADER]: Becoming Leader. State: Replica: 669e24a21f274a65b30ca62dbfb61a5e, State: Running, Role: LEADER
I20260808 04:47:57.471382 9591 consensus_queue.cc:237] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } }
I20260808 04:47:57.474772 8844 catalog_manager.cc:5495] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 669e24a21f274a65b30ca62dbfb61a5e (127.22.2.129). New cstate: current_term: 1 leader_uuid: "669e24a21f274a65b30ca62dbfb61a5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "669e24a21f274a65b30ca62dbfb61a5e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 } health_report { overall_health: UNKNOWN } } }
W20260808 04:47:57.496490 9137 tablet.cc:2369] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:47:57.498020 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.494013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56076 (local address 127.22.2.129:40295)
0808 04:47:57.494163 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 04:47:57.494168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:47:57.494234 (+ 66us) server_negotiation.cc:407] Connection header received
0808 04:47:57.494431 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:47:57.494434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:47:57.494494 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:47:57.494609 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:47:57.494873 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.495395 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.496296 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:47:57.496650 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:47:57.496762 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:47:57.497170 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:47:57.497585 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:47:57.497735 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 04:47:57.497799 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 04:47:57.497857 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 04:47:57.504066 9266 raft_consensus.cc:1261] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 1 FOLLOWER]: Refusing update from remote peer 669e24a21f274a65b30ca62dbfb61a5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:47:57.504098 9471 raft_consensus.cc:1261] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 1 FOLLOWER]: Refusing update from remote peer 669e24a21f274a65b30ca62dbfb61a5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:47:57.504874 9597 consensus_queue.cc:1035] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "19ee6cb573b748cca3e819b596407728" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:47:57.505024 9591 consensus_queue.cc:1035] T b1eed400c19c46fb81f254e4e9f8f6cd P 669e24a21f274a65b30ca62dbfb61a5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f58776b6b44a583e6097d2c8b1e3f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:47:57.517186 9606 mvcc.cc:205] Tried to move back new op lower bound from 7316129699849265152 to 7316129699725815808. Current Snapshot: MvccSnapshot[applied={T|T < 7316129699849265152}]
I20260808 04:47:57.517987 9604 mvcc.cc:205] Tried to move back new op lower bound from 7316129699849265152 to 7316129699725815808. Current Snapshot: MvccSnapshot[applied={T|T < 7316129699849265152}]
I20260808 04:47:57.521970 9605 mvcc.cc:205] Tried to move back new op lower bound from 7316129699849265152 to 7316129699725815808. Current Snapshot: MvccSnapshot[applied={T|T < 7316129699849265152}]
I20260808 04:47:57.526108 8843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:47:57.526288 8843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:47:57.529471 8843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66b111e14e2e4182bd1ec4146a72c851: Sending DeleteTablet for 3 replicas of tablet b1eed400c19c46fb81f254e4e9f8f6cd
I20260808 04:47:57.530652 9439 tablet_service.cc:1514] Processing DeleteTablet for tablet b1eed400c19c46fb81f254e4e9f8f6cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:47:57 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:48798
I20260808 04:47:57.530921 9237 tablet_service.cc:1514] Processing DeleteTablet for tablet b1eed400c19c46fb81f254e4e9f8f6cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:47:57 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:53990
I20260808 04:47:57.531143 9034 tablet_service.cc:1514] Processing DeleteTablet for tablet b1eed400c19c46fb81f254e4e9f8f6cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:47:57 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:56066
I20260808 04:47:57.531935 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 8905
I20260808 04:47:57.534741 9620 tablet_replica.cc:330] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: stopping tablet replica
I20260808 04:47:57.535166 9620 raft_consensus.cc:2229] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:47:57.535600 9620 raft_consensus.cc:2258] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:47:57.539492 9620 ts_tablet_manager.cc:1905] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:47:57.543869 9620 ts_tablet_manager.cc:1918] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:47:57.544059 9620 log.cc:1196] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/b1eed400c19c46fb81f254e4e9f8f6cd
I20260808 04:47:57.544575 9620 ts_tablet_manager.cc:1939] T b1eed400c19c46fb81f254e4e9f8f6cd P eb0f58776b6b44a583e6097d2c8b1e3f: Deleting consensus metadata
I20260808 04:47:57.546242 9619 tablet_replica.cc:330] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: stopping tablet replica
I20260808 04:47:57.546252 8824 catalog_manager.cc:4841] TS eb0f58776b6b44a583e6097d2c8b1e3f (127.22.2.130:36901): tablet b1eed400c19c46fb81f254e4e9f8f6cd (table test-table [id=fb1ff534d2a9485994ef8521dae8409b]) successfully deleted
I20260808 04:47:57.547978 9619 raft_consensus.cc:2229] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:47:57.548439 9619 raft_consensus.cc:2258] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:47:57.550437 9619 ts_tablet_manager.cc:1905] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:47:57.554287 9619 ts_tablet_manager.cc:1918] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:47:57.554450 9619 log.cc:1196] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/b1eed400c19c46fb81f254e4e9f8f6cd
I20260808 04:47:57.555047 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 9142
W20260808 04:47:57.555119 8824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:40295 (error 108)
I20260808 04:47:57.557061 9619 ts_tablet_manager.cc:1939] T b1eed400c19c46fb81f254e4e9f8f6cd P 19ee6cb573b748cca3e819b596407728: Deleting consensus metadata
I20260808 04:47:57.558296 8825 catalog_manager.cc:4841] TS 19ee6cb573b748cca3e819b596407728 (127.22.2.131:37707): tablet b1eed400c19c46fb81f254e4e9f8f6cd (table test-table [id=fb1ff534d2a9485994ef8521dae8409b]) successfully deleted
I20260808 04:47:57.579692 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:47:57.558924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:40295 (local address 127.0.0.1:56090)
0808 04:47:57.579487 (+ 20563us) negotiation.cc:107] Waiting for socket to connect
0808 04:47:57.579597 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:40295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20438}
W20260808 04:47:57.580559 8824 catalog_manager.cc:4584] TS 669e24a21f274a65b30ca62dbfb61a5e (127.22.2.129:40295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1eed400c19c46fb81f254e4e9f8f6cd: Network error: Client connection negotiation failed: client connection to 127.22.2.129:40295: connect: Connection refused (error 111)
I20260808 04:47:57.588953 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 9336
I20260808 04:47:57.600345 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 8794
I20260808 04:47:57.614239 22538 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 04:47:57.959079 22538 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 04:47:57.959230 22538 mini_ranger.cc:66] Stopping Ranger...
I20260808 04:47:58.303768 22538 mini_ranger.cc:68] Stopped Ranger
2026-08-08 04:47:58.303 UTC [7449] LOG: received smart shutdown request
2026-08-08 04:47:58.308 UTC [7449] LOG: background worker "logical replication launcher" (PID 7462) exited with exit code 1
2026-08-08 04:47:58.308 UTC [7456] LOG: shutting down
2026-08-08 04:47:58.483 UTC [7449] LOG: database system is shut down
2026-08-08T04:47:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49942 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:47:58 dist-test-slave-04bb krb5kdc[9637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:47:58 dist-test-slave-04bb krb5kdc[9637](info): set up 2 sockets
Aug 08 04:47:58 dist-test-slave-04bb krb5kdc[9637](info): commencing operation
krb5kdc: starting...
W20260808 04:47:59.857462 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:47:59 dist-test-slave-04bb krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:47:59.989602 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41257
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41257,127.22.2.189:41933,127.22.2.188:34463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:00.107941 9656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:00.108227 9656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:00.108307 9656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:00.112300 9656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:00.112385 9656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:00.112426 9656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:00.112464 9656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:00.112499 9656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:00.117720 9656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41257,127.22.2.189:41933,127.22.2.188:34463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:00.118791 9656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:00.119840 9656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:00.129287 9664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:00.129297 9667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:00.129320 9663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:00.129989 9656 server_base.cc:1034] running on GCE node
I20260808 04:48:00.130992 9656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:00.132670 9656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:00.133846 9656 hybrid_clock.cc:648] HybridClock initialized: now 1786164480133809 us; error 46 us; skew 500 ppm
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:00.137317 9656 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:48:00.140640 9656 webserver.cc:460] Webserver started at http://127.22.2.190:46835/ using document root <none> and password file <none>
I20260808 04:48:00.141419 9656 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:00.141492 9656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:00.141780 9656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:00.144381 9656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "648cbda02d68409f8f0b2819409355cd"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "7a4c6635eec1bfc18ce379f3098612a4"
server_key_iv: "c51f29c9dd61da6d04a17301b4e9d717"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.145033 9656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "648cbda02d68409f8f0b2819409355cd"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "7a4c6635eec1bfc18ce379f3098612a4"
server_key_iv: "c51f29c9dd61da6d04a17301b4e9d717"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.149089 9656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 04:48:00.151738 9678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:00.153126 9656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:48:00.153301 9656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "648cbda02d68409f8f0b2819409355cd"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "7a4c6635eec1bfc18ce379f3098612a4"
server_key_iv: "c51f29c9dd61da6d04a17301b4e9d717"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.153425 9656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:00.167766 9656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:00.171217 9656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:00.171446 9656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:00.179340 9656 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41257
I20260808 04:48:00.179322 9766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41257 every 8 connection(s)
I20260808 04:48:00.180346 9656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:48:00.183353 9768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:00.187170 9768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:48:00.188786 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 9656
I20260808 04:48:00.188944 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:48:00.189246 22538 external_mini_cluster.cc:1428] Setting key 50664c1fc4eb95eba6c953d923ac388e
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.189@KRBTEST.COM" created.
I20260808 04:48:00.203920 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.189574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.22.2.190:41257)
0808 04:48:00.190018 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.190048 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.190335 (+ 287us) server_negotiation.cc:407] Connection header received
0808 04:48:00.191347 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.191353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.191537 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.191728 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.192599 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.193469 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.194513 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.194880 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.195046 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.197995 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.198003 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.198017 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.198043 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.199891 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.200546 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.200556 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.200560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.200708 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.201204 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.201208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.201210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.201814 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.202000 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.202419 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.202712 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":294,"thread_start_us":142,"threads_started":1}
I20260808 04:48:00.203920 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.189575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41257 (local address 127.0.0.1:53050)
0808 04:48:00.190180 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.190217 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.191040 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.191728 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.191733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.192178 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.192428 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.192441 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.193595 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.193598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.194397 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.194407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.195046 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.195051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.195263 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.196006 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.196024 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.197811 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.200041 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.200051 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.200060 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.200391 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.200871 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.200876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.200882 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.201081 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.201965 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.201973 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.202137 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.202335 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.202773 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":184,"spinlock_wait_cycles":10624,"thread_start_us":164,"threads_started":1}
I20260808 04:48:00.206580 9768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:48:00.207451 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.207145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47616)
0808 04:48:00.207304 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.207384 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 04:48:00.207772 9679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
I20260808 04:48:00.208773 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.208459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47618)
0808 04:48:00.208661 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.208714 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260808 04:48:00.209030 9768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:41933: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
I20260808 04:48:00.220113 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:41933
--webserver_interface=127.22.2.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41257,127.22.2.189:41933,127.22.2.188:34463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:00.258299 9768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } attempt: 1
I20260808 04:48:00.259321 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.258948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47630)
0808 04:48:00.259119 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.259233 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 04:48:00.260149 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.259891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47644)
0808 04:48:00.260020 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.260082 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 04:48:00.260290 9768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:41933: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
I20260808 04:48:00.328472 9768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } attempt: 2
I20260808 04:48:00.329493 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.329102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47658)
0808 04:48:00.329301 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.329386 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 04:48:00.330336 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.330068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47670)
0808 04:48:00.330194 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.330253 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 04:48:00.330514 9768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:41933: Network error: Client connection negotiation failed: client connection to 127.22.2.189:41933: connect: Connection refused (error 111)
W20260808 04:48:00.346019 9781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:00.346263 9781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:00.346339 9781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:00.350440 9781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:00.350492 9781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:00.350512 9781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:00.350528 9781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:00.350570 9781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:00.355713 9781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41257,127.22.2.189:41933,127.22.2.188:34463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:41933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.2.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:00.356909 9781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:00.358176 9781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:00.366741 9791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:00.366752 9789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:00.366783 9795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:00.367199 9781 server_base.cc:1034] running on GCE node
I20260808 04:48:00.367781 9781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:00.369046 9781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:00.370290 9781 hybrid_clock.cc:648] HybridClock initialized: now 1786164480370274 us; error 38 us; skew 500 ppm
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:00.373353 9781 init.cc:377] Logged in from keytab as kudu/127.22.2.189@KRBTEST.COM (short username kudu)
I20260808 04:48:00.374394 9781 webserver.cc:460] Webserver started at http://127.22.2.189:40427/ using document root <none> and password file <none>
I20260808 04:48:00.374977 9781 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:00.375072 9781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:00.375325 9781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:00.377457 9781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/data/instance:
uuid: "59fc356e0ebb4aafa0c1b947b27a18cb"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "385d380eda1fd8a64ffe0e51c5c4a705"
server_key_iv: "75dfeba8ac40fa6c50b00c996af8a71c"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.378038 9781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/wal/instance:
uuid: "59fc356e0ebb4aafa0c1b947b27a18cb"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "385d380eda1fd8a64ffe0e51c5c4a705"
server_key_iv: "75dfeba8ac40fa6c50b00c996af8a71c"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.381919 9781 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:48:00.384418 9807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:00.385872 9781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:48:00.386022 9781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/wal
uuid: "59fc356e0ebb4aafa0c1b947b27a18cb"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "385d380eda1fd8a64ffe0e51c5c4a705"
server_key_iv: "75dfeba8ac40fa6c50b00c996af8a71c"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.386137 9781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:00.402449 9781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:00.405781 9781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:00.405990 9781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:00.413300 9781 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:41933
I20260808 04:48:00.413316 9885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:41933 every 8 connection(s)
I20260808 04:48:00.414254 9781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
I20260808 04:48:00.417168 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:00.417982 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 9781
I20260808 04:48:00.418118 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-1/wal/instance
I20260808 04:48:00.418447 22538 external_mini_cluster.cc:1428] Setting key 12771224f035f28c65d4247befee8d2f
I20260808 04:48:00.421736 9768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } attempt: 3
I20260808 04:48:00.422236 9886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.2.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.188@KRBTEST.COM" created.
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:00.437616 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.422295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47686)
0808 04:48:00.422531 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.422550 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.422653 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.425557 (+ 2904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.425560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.425774 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.425892 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.425899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.427361 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.427365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.428279 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.428290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.429034 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.429039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.429139 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.429695 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.429712 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.432558 (+ 2846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.435376 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.435380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.435382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.435579 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.435995 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.436001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.436006 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.436131 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.436925 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.436931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.437047 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.437174 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.437343 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260808 04:48:00.438364 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.424244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.22.2.190:41257)
0808 04:48:00.424358 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.424363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.424799 (+ 436us) server_negotiation.cc:407] Connection header received
0808 04:48:00.425368 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.425374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.425440 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.425542 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.426720 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.427417 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.428494 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.428950 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.429080 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.433441 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.433447 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.433453 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.433487 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.435177 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.435804 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.435807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.435808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.435882 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.436640 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.436644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.436645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.436859 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.436948 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.438065 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.438185 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 04:48:00.438710 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.423049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47686 (local address 127.22.2.189:41933)
0808 04:48:00.423578 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.423601 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.423647 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:48:00.425128 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.425151 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.425392 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.425579 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.426011 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.427222 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.428412 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.428908 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.429060 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.432733 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.432744 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.432762 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.432797 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.435283 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.435708 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.435714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.435716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.435883 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.436257 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.436259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.436260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.436795 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.436940 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.437280 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.437592 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":150,"threads_started":1}
I20260808 04:48:00.438910 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.424198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41257 (local address 127.0.0.1:53062)
0808 04:48:00.424630 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.424684 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.425128 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.425818 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.425827 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.426310 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.426537 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.426554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.427535 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.427538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.428294 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.428304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.429131 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.429135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.429263 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.429910 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.429926 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.433240 (+ 3314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.435323 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.435336 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.435349 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.435683 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.436285 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.436291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.436296 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.436404 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.437005 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.437010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.437127 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.438563 (+ 1436us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.438746 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":110,"threads_started":1}
I20260808 04:48:00.440236 9886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:48:00.440954 9768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:48:00.441584 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.441376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34840)
0808 04:48:00.441474 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.441527 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 04:48:00.442440 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.442118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34854)
0808 04:48:00.442298 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.442343 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260808 04:48:00.442631 9768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:34463: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 04:48:00.449204 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.440774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47696)
0808 04:48:00.440888 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.440903 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.440981 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.441329 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.441334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.441519 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.441650 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.441660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.442624 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.442630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.443330 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.443339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.443803 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.443809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.443936 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.444375 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.444386 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.445918 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.447643 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.447648 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.447650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.447951 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.448206 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.448209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.448210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.448356 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.448761 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.448763 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.448876 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.448998 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.449099 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 04:48:00.449369 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.440797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47696 (local address 127.22.2.189:41933)
0808 04:48:00.441112 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.441116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.441127 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:48:00.441154 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.441156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.441204 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.441286 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.441778 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.442473 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.443447 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.443687 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.443806 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.446059 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.446062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.446064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.446085 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.447522 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.448045 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.448048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.448049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.448122 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.448454 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.448456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.448457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.448671 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.448758 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.449065 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.449156 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260808 04:48:00.450084 9886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:48:00.450831 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.450590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34858)
0808 04:48:00.450703 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.450779 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
W20260808 04:48:00.451471 9810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
I20260808 04:48:00.451998 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:34463
--webserver_interface=127.22.2.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41257,127.22.2.189:41933,127.22.2.188:34463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:00.452375 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.452148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34870)
0808 04:48:00.452289 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.452328 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260808 04:48:00.452643 9886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:34463: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
I20260808 04:48:00.473814 9768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } attempt: 1
I20260808 04:48:00.474748 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.474438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34882)
0808 04:48:00.474601 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.474664 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 04:48:00.475581 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.475392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34886)
0808 04:48:00.475489 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.475532 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 04:48:00.475798 9768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:34463: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
I20260808 04:48:00.501930 9886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } attempt: 1
I20260808 04:48:00.502816 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.502509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34892)
0808 04:48:00.502669 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.502738 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 04:48:00.503610 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.503411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34904)
0808 04:48:00.503512 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.503558 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 04:48:00.503837 9886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:34463: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
I20260808 04:48:00.550976 9768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } attempt: 2
I20260808 04:48:00.551918 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.551592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34916)
0808 04:48:00.551765 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.551833 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 04:48:00.552767 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.552569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34928)
0808 04:48:00.552670 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.552715 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 04:48:00.552953 9768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:34463: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
I20260808 04:48:00.572016 9886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } attempt: 2
I20260808 04:48:00.572978 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.572684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34936)
0808 04:48:00.572840 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.572902 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 04:48:00.573714 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.573527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34940)
0808 04:48:00.573619 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.573662 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 04:48:00.573858 9886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:34463: Network error: Client connection negotiation failed: client connection to 127.22.2.188:34463: connect: Connection refused (error 111)
W20260808 04:48:00.574414 9895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:00.574765 9895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:00.574812 9895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:00.578972 9895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:00.579025 9895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:00.579044 9895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:00.579061 9895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:00.579077 9895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:00.584365 9895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41257,127.22.2.189:41933,127.22.2.188:34463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:34463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.2.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:00.585489 9895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:00.586498 9895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:00.595288 9902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:00.595417 9895 server_base.cc:1034] running on GCE node
W20260808 04:48:00.595347 9906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:00.595680 9915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:00.596262 9895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:00.597415 9895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:00.598631 9895 hybrid_clock.cc:648] HybridClock initialized: now 1786164480598612 us; error 31 us; skew 500 ppm
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:00.601451 9895 init.cc:377] Logged in from keytab as kudu/127.22.2.188@KRBTEST.COM (short username kudu)
I20260808 04:48:00.602298 9895 webserver.cc:460] Webserver started at http://127.22.2.188:36643/ using document root <none> and password file <none>
I20260808 04:48:00.602852 9895 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:00.602913 9895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:00.603082 9895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:00.605156 9895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/data/instance:
uuid: "6a5cd329ca3540d582ff168aaa337414"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "40acd1a6393cc8e204a36b588dfd3e1d"
server_key_iv: "804c19776dc3c80e1f54637794af2560"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.605682 9895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/wal/instance:
uuid: "6a5cd329ca3540d582ff168aaa337414"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "40acd1a6393cc8e204a36b588dfd3e1d"
server_key_iv: "804c19776dc3c80e1f54637794af2560"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.609541 9895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 04:48:00.611965 9930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:00.613329 9895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:48:00.613461 9895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/wal
uuid: "6a5cd329ca3540d582ff168aaa337414"
format_stamp: "Formatted at 2026-08-08 04:48:00 on dist-test-slave-04bb"
server_key: "40acd1a6393cc8e204a36b588dfd3e1d"
server_key_iv: "804c19776dc3c80e1f54637794af2560"
server_key_version: "encryptionkey@0"
I20260808 04:48:00.613576 9895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:00.628082 9895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:00.631165 9895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:00.631376 9895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:00.638257 9895 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:34463
I20260808 04:48:00.638374 10007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:34463 every 8 connection(s)
I20260808 04:48:00.639251 9895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
I20260808 04:48:00.639482 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 9895
I20260808 04:48:00.639627 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786164310373382-22538-0/minicluster-data/master-2/wal/instance
I20260808 04:48:00.639986 22538 external_mini_cluster.cc:1428] Setting key 6a86fb8c1316e2c82e894172a7d71437
I20260808 04:48:00.642825 10008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:00.647398 10008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:48:00.652122 9768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } attempt: 3
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:00.655870 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.641367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34948 (local address 127.22.2.188:34463)
0808 04:48:00.641792 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.641811 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.641853 (+ 42us) server_negotiation.cc:407] Connection header received
0808 04:48:00.642683 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.642706 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.643105 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.643461 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.644496 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.646072 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.646992 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.647674 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.647973 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.650679 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.650684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.650695 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.650727 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.652720 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.653244 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.653250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.653256 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.653411 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.653746 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.653749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.653751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.654234 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.654515 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.654805 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.655105 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":108,"threads_started":1}
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:48:00.659989 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.649433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.22.2.190:41257)
0808 04:48:00.649558 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.649563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.649958 (+ 395us) server_negotiation.cc:407] Connection header received
0808 04:48:00.650125 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.650128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.650188 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.650271 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.651094 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.651915 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.653131 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.653423 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.653542 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.656235 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.656239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.656241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.656267 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.657932 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.658543 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.658550 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.658555 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.658647 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.658995 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.659000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.659005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.659232 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.659354 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.659679 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.659795 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":48}
I20260808 04:48:00.661088 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.649400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41257 (local address 127.0.0.1:53070)
0808 04:48:00.649831 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.649869 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.650014 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.650323 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.650329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.650775 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.650947 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.650959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.652048 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.652051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.652997 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.653011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.653661 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.653665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.653797 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.654398 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.654412 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.656073 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.658107 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.658117 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.658124 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.658407 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.658789 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.658792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.658794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.658889 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.659353 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.659359 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.659478 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.659621 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.659799 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":167,"threads_started":1}
I20260808 04:48:00.662317 10008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:48:00.663478 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.652549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34956)
0808 04:48:00.652723 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.652738 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.652816 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.653258 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.653262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.653481 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.653680 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.653690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.654712 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.654716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.655599 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.655613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.656254 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.656258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.656339 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.656939 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.656956 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.659582 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.661784 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.661787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.661789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.661998 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.662294 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.662297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.662299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.662397 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.662887 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.662893 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.663011 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.663138 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.663305 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":55}
I20260808 04:48:00.664012 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.652628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.22.2.188:34463)
0808 04:48:00.653017 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.653023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.653033 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:48:00.653060 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.653061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.653125 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.653208 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.653842 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.654571 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.655756 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.656122 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.656225 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.659736 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.659741 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.659744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.659774 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.661683 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.662104 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.662108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.662109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.662196 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.662515 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.662519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.662522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.662754 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.662856 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.663704 (+ 848us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.663815 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":89,"threads_started":1}
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:00.665038 9886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } attempt: 3
I20260808 04:48:00.668032 9768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd: Bootstrap starting.
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:48:00.671453 9768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:00.672478 9768 log.cc:824] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:00.673157 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.658519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53084 (local address 127.22.2.190:41257)
0808 04:48:00.658787 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.658799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.658827 (+ 28us) server_negotiation.cc:407] Connection header received
0808 04:48:00.658861 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.658864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.658906 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.659050 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.659785 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.660564 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.661537 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.661883 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.661974 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.664770 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.664774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.664777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.664811 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.669889 (+ 5078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.670445 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.670449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.670451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.670542 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.670897 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.670902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.670906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.671141 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.671355 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.671885 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.672914 (+ 1029us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":76,"threads_started":1}
I20260808 04:48:00.673532 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.662793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:41933 (local address 127.0.0.1:47700)
0808 04:48:00.663074 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.663087 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.663159 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.663432 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.663435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.663617 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.663744 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.663752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.664772 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.664775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.665815 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.665827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.666557 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.666561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.666651 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.667090 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.667110 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.668510 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.671375 (+ 2865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.671381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.671384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.671948 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.672286 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.672288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.672290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.672381 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.673027 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.673030 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.673129 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.673246 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.673375 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260808 04:48:00.673563 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.663077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47700 (local address 127.22.2.189:41933)
0808 04:48:00.663216 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.663222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.663236 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:48:00.663279 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.663287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.663326 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.663419 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.663892 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.664642 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.665949 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.666323 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.666408 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.668687 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.668691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.668693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.668719 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.670974 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.672068 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.672071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.672074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.672161 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.672490 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.672497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.672500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.672859 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.672948 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.673308 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.673418 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56,"spinlock_wait_cycles":5376}
I20260808 04:48:00.674214 10008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:48:00.675012 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.665442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34962)
0808 04:48:00.665567 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.665583 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.665678 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.665893 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.665896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.666062 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.666192 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.666200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.667191 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.667196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.667903 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.667913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.668472 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.668475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.668590 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.669018 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.669031 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.670681 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.672692 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.672694 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.672696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.673030 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.673944 (+ 914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.673948 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.673950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.674045 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.674500 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.674503 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.674599 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.674706 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.674843 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 04:48:00.675506 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.665474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.22.2.188:34463)
0808 04:48:00.665566 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.665570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.665639 (+ 69us) server_negotiation.cc:407] Connection header received
0808 04:48:00.665746 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.665749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.665780 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.665871 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.666362 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.667052 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.668029 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.668353 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.668467 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.670834 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.670840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.670847 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.670875 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.672515 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.673154 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.673158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.673160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.673490 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.674162 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.674165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.674166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.674405 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.674503 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.675209 (+ 706us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.675336 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:48:00.678267 9768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd: No bootstrap required, opened a new log
I20260808 04:48:00.678691 9886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb: Bootstrap starting.
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164480, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:48:00.681746 9886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb: Neither blocks nor log segments found. Creating new log.
Aug 08 04:48:00 dist-test-slave-04bb krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 04:48:00.682672 9886 log.cc:824] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:00.683048 9768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.683310 9768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:00.683382 9768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648cbda02d68409f8f0b2819409355cd, State: Initialized, Role: FOLLOWER
I20260808 04:48:00.683610 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.674913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.22.2.188:34463)
0808 04:48:00.675042 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.675046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.675177 (+ 131us) server_negotiation.cc:407] Connection header received
0808 04:48:00.675357 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.675361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.675407 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.675462 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.676007 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.676749 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.677726 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.678028 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.678102 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.680281 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.680285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.680287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.680321 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.681926 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.682417 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.682421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.682423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.682504 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.682765 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.682768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.682769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.682949 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.683017 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.683342 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.683469 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 04:48:00.683990 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.674709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:34463 (local address 127.0.0.1:34970)
0808 04:48:00.675114 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:00.675128 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:00.675240 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:00.675600 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:00.675603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:00.675792 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:00.675890 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.675898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.676886 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.676892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.677602 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:00.677614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.678227 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.678230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:00.678320 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.678747 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:00.678762 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:00.680151 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:00.682037 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.682040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.682042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.682273 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.682603 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:00.682605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:00.682607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:00.682682 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:00.683078 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:00.683081 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:00.683161 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 04:48:00.683732 (+ 571us) client_negotiation.cc:240] Negotiation successful
0808 04:48:00.683853 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349}
I20260808 04:48:00.683940 9768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.684435 10026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.684587 10026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:48:00.685249 9768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:00.687135 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.674679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47714 (local address 127.22.2.189:41933)
0808 04:48:00.674784 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.674788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.676152 (+ 1364us) server_negotiation.cc:407] Connection header received
0808 04:48:00.676246 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.676249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.676291 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.676366 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.677211 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.677898 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.679168 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.679456 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.679549 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.682950 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.682954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.682956 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.682980 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.685038 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.685506 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.685509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.685511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.685597 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.685917 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.685920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.685921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.686124 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.686287 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.686897 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.687003 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 04:48:00.687139 10008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414: Bootstrap starting.
I20260808 04:48:00.690456 10008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414: Neither blocks nor log segments found. Creating new log.
W20260808 04:48:00.690864 10041 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:48:00.690935 10041 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:48:00.691269 10008 log.cc:824] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:00.693001 9886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb: No bootstrap required, opened a new log
I20260808 04:48:00.693049 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:48:00.693920 10008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414: No bootstrap required, opened a new log
I20260808 04:48:00.690027 9656 master_runner.cc:186] Error getting master registration for 127.22.2.189:41933: OK, instance_id { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" instance_seqno: 1786164480411237 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 04:48:00.695979 9656 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 04:48:00.697067 9886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.697357 9886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:00.697422 9886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59fc356e0ebb4aafa0c1b947b27a18cb, State: Initialized, Role: FOLLOWER
I20260808 04:48:00.697925 9886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.698191 10008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.698467 10008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:00.698556 10008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5cd329ca3540d582ff168aaa337414, State: Initialized, Role: FOLLOWER
I20260808 04:48:00.699030 10008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.699316 9886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:00.700444 10008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:00.708050 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.696399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.22.2.188:34463)
0808 04:48:00.696508 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.696515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.698469 (+ 1954us) server_negotiation.cc:407] Connection header received
0808 04:48:00.698603 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.698606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.698656 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.698734 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.699679 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.700459 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.701700 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.702099 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.702203 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.704117 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.704123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.704129 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.704161 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.706162 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.706646 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.706649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.706651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.706750 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.707077 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.707081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.707082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.707292 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.707412 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.707746 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.707884 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 04:48:00.708830 9781 master_runner.cc:263] No leader master found from master 59fc356e0ebb4aafa0c1b947b27a18cb
I20260808 04:48:00.708899 9781 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 04:48:00.709807 9895 master_runner.cc:263] No leader master found from master 6a5cd329ca3540d582ff168aaa337414
I20260808 04:48:00.709868 9895 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 04:48:00.711692 10080 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:48:00.711767 10080 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:48:00.713088 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.713220 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:48:00.713510 10060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.713621 10060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:48:00.714553 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.697140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53086 (local address 127.22.2.190:41257)
0808 04:48:00.697247 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.697251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.704809 (+ 7558us) server_negotiation.cc:407] Connection header received
0808 04:48:00.704991 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.704994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.705043 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.705094 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.705967 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.706680 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.707728 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.708085 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.708176 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.710334 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.710338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.710341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.710367 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.712187 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.712956 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.712960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.712963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.713058 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.713476 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.713480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.713481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.713711 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.713867 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.714285 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.714409 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 04:48:00.716987 10091 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:48:00.717062 10091 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:48:00.719345 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:00.697786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.22.2.189:41933)
0808 04:48:00.697905 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:00.697908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:00.709026 (+ 11118us) server_negotiation.cc:407] Connection header received
0808 04:48:00.709132 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:00.709138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:00.709192 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:00.709263 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:00.710196 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.710940 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.712086 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:00.712424 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:00.712500 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:00.714836 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:00.714841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:00.714843 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:00.714870 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:00.717293 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.717911 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.717915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.717916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.718009 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:00.718357 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:00.718361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:00.718363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:00.718573 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:00.718672 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:00.719065 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 04:48:00.719186 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 04:48:00.807121 10060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:00.807420 10060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.809316 9720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a5cd329ca3540d582ff168aaa337414" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648cbda02d68409f8f0b2819409355cd" is_pre_election: true
I20260808 04:48:00.809847 9720 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5cd329ca3540d582ff168aaa337414 in term 0.
I20260808 04:48:00.810417 9933 leader_election.cc:302] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648cbda02d68409f8f0b2819409355cd, 6a5cd329ca3540d582ff168aaa337414; no voters:
I20260808 04:48:00.811538 9851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a5cd329ca3540d582ff168aaa337414" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" is_pre_election: true
I20260808 04:48:00.811851 9851 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5cd329ca3540d582ff168aaa337414 in term 0.
I20260808 04:48:00.812330 10060 leader_election.cc:288] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 648cbda02d68409f8f0b2819409355cd (127.22.2.190:41257), 59fc356e0ebb4aafa0c1b947b27a18cb (127.22.2.189:41933)
I20260808 04:48:00.812654 10060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:48:00.812744 10060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:48:00.812860 10060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:00.814177 10060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.819885 9720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a5cd329ca3540d582ff168aaa337414" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648cbda02d68409f8f0b2819409355cd"
I20260808 04:48:00.820036 9720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:00.821496 9720 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5cd329ca3540d582ff168aaa337414 in term 1.
I20260808 04:48:00.821889 9933 leader_election.cc:302] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648cbda02d68409f8f0b2819409355cd, 6a5cd329ca3540d582ff168aaa337414; no voters:
I20260808 04:48:00.822518 9851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a5cd329ca3540d582ff168aaa337414" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb"
I20260808 04:48:00.822671 9851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:00.824066 9851 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5cd329ca3540d582ff168aaa337414 in term 1.
I20260808 04:48:00.824493 10060 leader_election.cc:288] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [CANDIDATE]: Term 1 election: Requested vote from peers 648cbda02d68409f8f0b2819409355cd (127.22.2.190:41257), 59fc356e0ebb4aafa0c1b947b27a18cb (127.22.2.189:41933)
I20260808 04:48:00.824707 10060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:48:00.824869 10060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5cd329ca3540d582ff168aaa337414, State: Running, Role: LEADER
I20260808 04:48:00.825268 10060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } }
I20260808 04:48:00.827339 10060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a5cd329ca3540d582ff168aaa337414. Latest consensus state: current_term: 1 leader_uuid: "6a5cd329ca3540d582ff168aaa337414" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.827481 10060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:00.828017 10099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:48:00.829157 10099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:48:00.839582 9851 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [term 1 FOLLOWER]: Refusing update from remote peer 6a5cd329ca3540d582ff168aaa337414: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:00.840301 10097 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:00.844303 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a5cd329ca3540d582ff168aaa337414. Latest consensus state: current_term: 1 leader_uuid: "6a5cd329ca3540d582ff168aaa337414" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.844429 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:48:00.845463 9720 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [term 1 FOLLOWER]: Refusing update from remote peer 6a5cd329ca3540d582ff168aaa337414: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:00.846079 10097 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:00.850268 10026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a5cd329ca3540d582ff168aaa337414. Latest consensus state: current_term: 1 leader_uuid: "6a5cd329ca3540d582ff168aaa337414" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.850407 10026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:48:00.858740 10097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a5cd329ca3540d582ff168aaa337414" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.858906 10097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:00.860189 10026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a5cd329ca3540d582ff168aaa337414" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.860360 10026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:48:00.860781 10100 mvcc.cc:205] Tried to move back new op lower bound from 7316129713511157760 to 7316129713464254464. Current Snapshot: MvccSnapshot[applied={T|T < 7316129713511157760}]
I20260808 04:48:00.864094 10107 mvcc.cc:205] Tried to move back new op lower bound from 7316129713511157760 to 7316129713464254464. Current Snapshot: MvccSnapshot[applied={T|T < 7316129713511157760}]
I20260808 04:48:00.864986 10060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a5cd329ca3540d582ff168aaa337414" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.865110 10060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414 [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:00.866501 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a5cd329ca3540d582ff168aaa337414" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648cbda02d68409f8f0b2819409355cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41257 } } peers { permanent_uuid: "59fc356e0ebb4aafa0c1b947b27a18cb" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 41933 } } peers { permanent_uuid: "6a5cd329ca3540d582ff168aaa337414" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 34463 } } }
I20260808 04:48:00.866616 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:48:00.867414 10103 mvcc.cc:205] Tried to move back new op lower bound from 7316129713511157760 to 7316129713464254464. Current Snapshot: MvccSnapshot[applied={T|T < 7316129713511157760}]
I20260808 04:48:00.875244 10099 catalog_manager.cc:1347] Generated new cluster ID: 8c7214fc6a774fc88fe21520d9a4f086
I20260808 04:48:00.875326 10099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:48:00.907658 10099 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:48:00.908878 10099 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:48:00.917997 10099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a5cd329ca3540d582ff168aaa337414: Generated new TSK 0
I20260808 04:48:00.919378 10099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:48:01.692343 10041 catalog_manager.cc:1259] Loaded cluster ID: 8c7214fc6a774fc88fe21520d9a4f086
I20260808 04:48:01.692430 10041 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd: loading cluster ID for follower catalog manager: success
I20260808 04:48:01.694257 10041 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd: acquiring CA information for follower catalog manager: success
I20260808 04:48:01.695472 10041 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 648cbda02d68409f8f0b2819409355cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 04:48:01.713050 10080 catalog_manager.cc:1259] Loaded cluster ID: 8c7214fc6a774fc88fe21520d9a4f086
I20260808 04:48:01.713156 10080 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb: loading cluster ID for follower catalog manager: success
I20260808 04:48:01.714795 10080 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb: acquiring CA information for follower catalog manager: success
I20260808 04:48:01.715951 10080 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59fc356e0ebb4aafa0c1b947b27a18cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 04:48:01.987169 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 9656
I20260808 04:48:02.001516 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 9781
I20260808 04:48:02.012873 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 9895
2026-08-08T04:48:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3417 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:48:02 dist-test-slave-04bb krb5kdc[10122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:48:02 dist-test-slave-04bb krb5kdc[10122](info): set up 2 sockets
Aug 08 04:48:02 dist-test-slave-04bb krb5kdc[10122](info): commencing operation
krb5kdc: starting...
W20260808 04:48:03.254686 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:48:03 dist-test-slave-04bb krb5kdc[10122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:48:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:48:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:48:03.384027 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45607
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:45607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:03.499442 10138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:03.499787 10138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:03.499827 10138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:03.503768 10138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:03.503819 10138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:03.503839 10138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:03.503854 10138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:03.503870 10138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:03.508965 10138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:45607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:03.509991 10138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:03.510957 10138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:03.519650 10147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:03.519726 10151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:03.519793 10138 server_base.cc:1034] running on GCE node
W20260808 04:48:03.519650 10148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:03.520560 10138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:03.521610 10138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:03.522812 10138 hybrid_clock.cc:648] HybridClock initialized: now 1786164483522781 us; error 44 us; skew 500 ppm
Aug 08 04:48:03 dist-test-slave-04bb krb5kdc[10122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164483, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:03.529911 10138 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:48:03.530891 10138 webserver.cc:460] Webserver started at http://127.22.2.190:43055/ using document root <none> and password file <none>
I20260808 04:48:03.531487 10138 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:03.531580 10138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:03.531823 10138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:03.533949 10138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "76bc820a5477475a9dcd6c7fb247010a"
format_stamp: "Formatted at 2026-08-08 04:48:03 on dist-test-slave-04bb"
server_key: "f2ea5f5e48a1107f3b08097a648cbc93"
server_key_iv: "c72fb52262d3cda7323eca649ce28f05"
server_key_version: "encryptionkey@0"
I20260808 04:48:03.534531 10138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "76bc820a5477475a9dcd6c7fb247010a"
format_stamp: "Formatted at 2026-08-08 04:48:03 on dist-test-slave-04bb"
server_key: "f2ea5f5e48a1107f3b08097a648cbc93"
server_key_iv: "c72fb52262d3cda7323eca649ce28f05"
server_key_version: "encryptionkey@0"
I20260808 04:48:03.538290 10138 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260808 04:48:03.540688 10158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:03.542014 10138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:48:03.542157 10138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "76bc820a5477475a9dcd6c7fb247010a"
format_stamp: "Formatted at 2026-08-08 04:48:03 on dist-test-slave-04bb"
server_key: "f2ea5f5e48a1107f3b08097a648cbc93"
server_key_iv: "c72fb52262d3cda7323eca649ce28f05"
server_key_version: "encryptionkey@0"
I20260808 04:48:03.542282 10138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:03.563431 10138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:03.566246 10138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:03.566453 10138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:03.573807 10138 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:45607
I20260808 04:48:03.573824 10233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:45607 every 8 connection(s)
I20260808 04:48:03.574781 10138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:48:03.577749 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:03.580930 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 10138
I20260808 04:48:03.581017 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:48:03.581351 22538 external_mini_cluster.cc:1428] Setting key d8c07574628b3a55112223504ea696b9
I20260808 04:48:03.583765 10234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a: Bootstrap starting.
I20260808 04:48:03.586050 10234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:03.586788 10234 log.cc:824] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:03.588830 10234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a: No bootstrap required, opened a new log
Aug 08 04:48:03 dist-test-slave-04bb krb5kdc[10122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:03.592396 10234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bc820a5477475a9dcd6c7fb247010a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45607 } }
I20260808 04:48:03.592697 10234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:03.592782 10234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76bc820a5477475a9dcd6c7fb247010a, State: Initialized, Role: FOLLOWER
I20260808 04:48:03.593247 10234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bc820a5477475a9dcd6c7fb247010a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45607 } }
I20260808 04:48:03.593410 10234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:48:03.593511 10234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:48:03.593641 10234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:03.594664 10234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bc820a5477475a9dcd6c7fb247010a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45607 } }
I20260808 04:48:03.595047 10234 leader_election.cc:302] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76bc820a5477475a9dcd6c7fb247010a; no voters:
I20260808 04:48:03.595412 10234 leader_election.cc:288] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:48:03.595503 10240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:48:03.595790 10240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [term 1 LEADER]: Becoming Leader. State: Replica: 76bc820a5477475a9dcd6c7fb247010a, State: Running, Role: LEADER
I20260808 04:48:03.596138 10240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bc820a5477475a9dcd6c7fb247010a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45607 } }
I20260808 04:48:03.596738 10234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:03.597313 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:03.582840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.22.2.190:45607)
0808 04:48:03.583309 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:03.583324 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:03.583360 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:48:03.584050 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:03.584069 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:03.584390 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:03.584799 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:03.585596 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:03.586661 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:03.587592 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:03.588075 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:03.588248 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:03.591300 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:03.591306 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:03.591320 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:03.591348 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:03.594047 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:03.594551 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:03.594556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:03.594561 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:03.594668 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:03.594996 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:03.594999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:03.595000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:03.595505 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:03.595725 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:03.596033 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 04:48:03.596337 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":131,"threads_started":1}
I20260808 04:48:03.597431 10242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76bc820a5477475a9dcd6c7fb247010a. Latest consensus state: current_term: 1 leader_uuid: "76bc820a5477475a9dcd6c7fb247010a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bc820a5477475a9dcd6c7fb247010a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45607 } } }
I20260808 04:48:03.597553 10242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:03.597949 10241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76bc820a5477475a9dcd6c7fb247010a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bc820a5477475a9dcd6c7fb247010a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45607 } } }
I20260808 04:48:03.598033 10241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:03.601123 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 04:48:03.601686 10261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:48:03.601783 10261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:48:03.601897 10249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:48:03.602821 10249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:48:03.608203 10249 catalog_manager.cc:1347] Generated new cluster ID: dde47d370617437d873928519377e26f
I20260808 04:48:03.608266 10249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:48:03.625890 10249 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:48:03.626909 10249 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:48:03.633538 10249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76bc820a5477475a9dcd6c7fb247010a: Generated new TSK 0
I20260808 04:48:03.634174 10249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:48:03.657204 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 10138
2026-08-08T04:48:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1641 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:48:03 dist-test-slave-04bb krb5kdc[10274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:48:03 dist-test-slave-04bb krb5kdc[10274](info): set up 2 sockets
Aug 08 04:48:03 dist-test-slave-04bb krb5kdc[10274](info): commencing operation
krb5kdc: starting...
W20260808 04:48:04.898413 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:48:04 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:48:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:48:05.029474 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42233
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:05.143939 10296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:05.144397 10296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:05.144549 10296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:05.152465 10296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:05.152572 10296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:05.152639 10296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:05.152678 10296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:05.152734 10296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:05.158862 10296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:05.159991 10296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:05.161116 10296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:05.169710 10304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:05.169723 10301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:05.169843 10302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:05.170423 10296 server_base.cc:1034] running on GCE node
I20260808 04:48:05.170986 10296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:05.172078 10296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:05.173269 10296 hybrid_clock.cc:648] HybridClock initialized: now 1786164485173248 us; error 46 us; skew 500 ppm
Aug 08 04:48:05 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164485, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:05.176113 10296 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:48:05.177145 10296 webserver.cc:460] Webserver started at http://127.22.2.190:35517/ using document root <none> and password file <none>
I20260808 04:48:05.177745 10296 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:05.177840 10296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:05.178081 10296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:05.180142 10296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "e54513a6d81e4608b09252c2b95ec61b"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "c98349ddc74df5d1e8337b059b0161c6"
server_key_iv: "fd8f1b3b9dfb5e92251ed41c862cdfa4"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.180812 10296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "e54513a6d81e4608b09252c2b95ec61b"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "c98349ddc74df5d1e8337b059b0161c6"
server_key_iv: "fd8f1b3b9dfb5e92251ed41c862cdfa4"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.184796 10296 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:48:05.187359 10315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.188614 10296 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:48:05.188764 10296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "e54513a6d81e4608b09252c2b95ec61b"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "c98349ddc74df5d1e8337b059b0161c6"
server_key_iv: "fd8f1b3b9dfb5e92251ed41c862cdfa4"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.188884 10296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:05.202849 10296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:05.206182 10296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:05.206390 10296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:05.213879 10296 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42233
I20260808 04:48:05.213894 10390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42233 every 8 connection(s)
I20260808 04:48:05.214859 10296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:48:05.215749 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 10296
I20260808 04:48:05.215873 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:48:05.216212 22538 external_mini_cluster.cc:1428] Setting key e3a963f7ed67dffbc219512fb12b4bec
I20260808 04:48:05.218357 10391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:05.225152 10391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Bootstrap starting.
Aug 08 04:48:05 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:05.227399 10391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:05.228144 10391 log.cc:824] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:05.230149 10391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: No bootstrap required, opened a new log
I20260808 04:48:05.231079 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.217648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54996 (local address 127.22.2.190:42233)
0808 04:48:05.218066 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:05.218083 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:05.218119 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:48:05.218829 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:05.218848 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:05.219180 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:05.219468 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:05.220419 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.221477 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.222631 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.223079 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.223205 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.226150 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:05.226155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:05.226168 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:05.226201 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:05.228010 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.228548 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.228558 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.228566 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.228695 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.228990 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.228993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.228995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.229378 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:05.229580 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:05.229854 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 04:48:05.230127 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":146,"threads_started":1}
I20260808 04:48:05.233161 10391 raft_consensus.cc:348] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:05.233436 10391 raft_consensus.cc:374] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:05.233531 10391 raft_consensus.cc:729] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54513a6d81e4608b09252c2b95ec61b, State: Initialized, Role: FOLLOWER
I20260808 04:48:05.233960 10391 consensus_queue.cc:260] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:05.234113 10391 raft_consensus.cc:388] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:48:05.234203 10391 raft_consensus.cc:482] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:48:05.234304 10391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:05.235147 10391 raft_consensus.cc:504] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:05.235502 10391 leader_election.cc:302] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e54513a6d81e4608b09252c2b95ec61b; no voters:
I20260808 04:48:05.235838 10391 leader_election.cc:288] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:48:05.235921 10398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:48:05.236236 10398 raft_consensus.cc:686] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 1 LEADER]: Becoming Leader. State: Replica: e54513a6d81e4608b09252c2b95ec61b, State: Running, Role: LEADER
I20260808 04:48:05.236649 10398 consensus_queue.cc:237] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:05.237118 10391 sys_catalog.cc:564] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:05.238289 10401 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e54513a6d81e4608b09252c2b95ec61b. Latest consensus state: current_term: 1 leader_uuid: "e54513a6d81e4608b09252c2b95ec61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } } }
I20260808 04:48:05.238454 10401 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:05.238833 10411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:48:05.239181 10400 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e54513a6d81e4608b09252c2b95ec61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } } }
I20260808 04:48:05.239285 10400 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:05.241967 10411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:48:05.248214 10411 catalog_manager.cc:1347] Generated new cluster ID: b07eea6058584cf5a0de91d0029c5b94
I20260808 04:48:05.248282 10411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:48:05.263267 10411 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:48:05.264139 10411 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:48:05.268239 10411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Generated new TSK 0
I20260808 04:48:05.268932 10411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:48:05.299837 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42233
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:05.413816 10428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:05.414090 10428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:05.414167 10428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:05.418146 10428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:05.418231 10428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:05.418329 10428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:48:05.423396 10428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:05.424382 10428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:05.425436 10428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:05.435031 10439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:05.435040 10437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:05.435138 10433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:05.435317 10428 server_base.cc:1034] running on GCE node
I20260808 04:48:05.435778 10428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:05.436398 10428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:05.437556 10428 hybrid_clock.cc:648] HybridClock initialized: now 1786164485437542 us; error 26 us; skew 500 ppm
Aug 08 04:48:05 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164485, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:05.440349 10428 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:48:05.441272 10428 webserver.cc:460] Webserver started at http://127.22.2.129:36241/ using document root <none> and password file <none>
I20260808 04:48:05.441838 10428 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:05.441927 10428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:05.442157 10428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:05.444125 10428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "ccd20a855e0e3100df1b3084988ee93a"
server_key_iv: "b4a86bc8ccfee5e30eced0cf77b390f5"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.444742 10428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "ccd20a855e0e3100df1b3084988ee93a"
server_key_iv: "b4a86bc8ccfee5e30eced0cf77b390f5"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.448304 10428 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 04:48:05.450686 10454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.451920 10428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:48:05.452052 10428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "ccd20a855e0e3100df1b3084988ee93a"
server_key_iv: "b4a86bc8ccfee5e30eced0cf77b390f5"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.452157 10428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:05.474289 10428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:05.477604 10428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:05.477820 10428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:05.478415 10428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:05.479233 10428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:05.479310 10428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.479379 10428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:05.479429 10428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.488909 10428 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:45129
I20260808 04:48:05.488932 10613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:45129 every 8 connection(s)
I20260808 04:48:05.489777 10428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:48:05.496807 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 10428
I20260808 04:48:05.496963 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:48:05.497232 22538 external_mini_cluster.cc:1428] Setting key e6f820af74241b2af5311aaeb2a4c310
Aug 08 04:48:05 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164485, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:05.504106 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.491616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:36181 (local address 127.22.2.190:42233)
0808 04:48:05.491759 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:05.491763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:05.492625 (+ 862us) server_negotiation.cc:407] Connection header received
0808 04:48:05.494123 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:05.494127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:05.494186 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:05.494297 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:05.495458 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.496004 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.497322 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.497588 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.497704 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.500444 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:05.500448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:05.500451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:05.500483 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:05.502045 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.502569 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.502573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.502575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.502659 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.503004 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.503009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.503010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.503195 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:05.503304 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:05.503795 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 04:48:05.503938 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:48:05.504905 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.491991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:36181)
0808 04:48:05.492448 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:05.492491 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:05.493909 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:05.494483 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:05.494489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:05.494831 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:05.495294 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.495316 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.496138 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.496142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.497195 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.497206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.497773 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.497803 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.498002 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.498660 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:05.498682 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:05.500274 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:05.502171 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:05.502177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:05.502192 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:05.502459 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:05.502754 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:05.502757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:05.502759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:05.502903 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:05.503316 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:05.503324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:05.503557 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 04:48:05.503745 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 04:48:05.504052 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":119,"threads_started":1}
I20260808 04:48:05.506242 10614 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42233
I20260808 04:48:05.506484 10614 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:05.506958 10614 heartbeater.cc:511] Master 127.22.2.190:42233 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:48:05.508565 10341 ts_manager.cc:194] Registered new tserver with Master: 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129:45129)
I20260808 04:48:05.509779 10341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:36181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:48:05.527127 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42233
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:05.645808 10626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:05.646085 10626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:05.646236 10626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:05.650652 10626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:05.650728 10626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:05.650815 10626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:48:05.655894 10626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:05.656950 10626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:05.657943 10626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:05.667351 10632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:05.667412 10636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:05.667419 10633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:05.667860 10626 server_base.cc:1034] running on GCE node
I20260808 04:48:05.668246 10626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:05.668908 10626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:05.670068 10626 hybrid_clock.cc:648] HybridClock initialized: now 1786164485670056 us; error 25 us; skew 500 ppm
Aug 08 04:48:05 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164485, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:05.672797 10626 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:48:05.673668 10626 webserver.cc:460] Webserver started at http://127.22.2.130:37487/ using document root <none> and password file <none>
I20260808 04:48:05.674192 10626 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:05.674250 10626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:05.674430 10626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:05.676409 10626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "395be1bb74ad4137a45158f22ce64607"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "dcc2a36559708428a9388f201ba5b63f"
server_key_iv: "890c6b09a840e99e537f052df92b9a96"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.677002 10626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "395be1bb74ad4137a45158f22ce64607"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "dcc2a36559708428a9388f201ba5b63f"
server_key_iv: "890c6b09a840e99e537f052df92b9a96"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.680645 10626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:48:05.683035 10650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.684346 10626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:48:05.684463 10626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "395be1bb74ad4137a45158f22ce64607"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "dcc2a36559708428a9388f201ba5b63f"
server_key_iv: "890c6b09a840e99e537f052df92b9a96"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.684617 10626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:05.703974 10626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:05.706772 10626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:05.706975 10626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:05.707556 10626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:05.708451 10626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:05.708510 10626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.708590 10626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:05.708629 10626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.717813 10626 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:39707
I20260808 04:48:05.717837 10820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:39707 every 8 connection(s)
I20260808 04:48:05.718685 10626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:48:05.724273 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 10626
I20260808 04:48:05.724399 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:48:05.724702 22538 external_mini_cluster.cc:1428] Setting key f6e8894f735aae028312a50a318f9c15
Aug 08 04:48:05 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164485, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:05.732685 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.720519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:49049 (local address 127.22.2.190:42233)
0808 04:48:05.720705 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:05.720710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:05.721503 (+ 793us) server_negotiation.cc:407] Connection header received
0808 04:48:05.722797 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:05.722800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:05.722849 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:05.722924 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:05.724044 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.724726 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.725891 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.726130 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.726246 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.728821 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:05.728825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:05.728827 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:05.728850 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:05.730546 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.731081 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.731084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.731086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.731172 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.731534 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.731538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.731540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.731731 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:05.731845 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:05.732365 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 04:48:05.732490 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 04:48:05.733482 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.720906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:49049)
0808 04:48:05.721345 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:05.721387 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:05.722595 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:05.723088 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:05.723095 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:05.723436 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:05.723909 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.723922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.724882 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.724885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.725750 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.725761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.726296 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.726301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.726467 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.727097 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:05.727113 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:05.728644 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:05.730674 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:05.730680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:05.730696 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:05.730965 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:05.731274 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:05.731280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:05.731283 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:05.731432 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:05.731845 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:05.731853 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:05.732116 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 04:48:05.732315 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 04:48:05.732659 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":119,"threads_started":1}
I20260808 04:48:05.734732 10821 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42233
I20260808 04:48:05.735000 10821 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:48:05.735601 10821 heartbeater.cc:511] Master 127.22.2.190:42233 requested a full tablet report, sending...
I20260808 04:48:05.736799 10341 ts_manager.cc:194] Registered new tserver with Master: 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707)
I20260808 04:48:05.737571 10341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:49049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:48:05.755378 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42233
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:05.871533 10830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:05.871794 10830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:05.871882 10830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:05.875911 10830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:05.875974 10830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:05.876070 10830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:48:05.881425 10830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:05.882546 10830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:05.883711 10830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:05.894164 10845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:05.894170 10850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:05.894241 10842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:05.894430 10830 server_base.cc:1034] running on GCE node
I20260808 04:48:05.894930 10830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:05.895603 10830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:05.896767 10830 hybrid_clock.cc:648] HybridClock initialized: now 1786164485896733 us; error 44 us; skew 500 ppm
Aug 08 04:48:05 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164485, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:05.899528 10830 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:48:05.900453 10830 webserver.cc:460] Webserver started at http://127.22.2.131:46841/ using document root <none> and password file <none>
I20260808 04:48:05.901069 10830 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:05.901139 10830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:05.901382 10830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:05.903401 10830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "b54e62f5b24844fe8385bd18754d3919"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "fdc86abbe8b35d1fe46fd5c1ef1cab9e"
server_key_iv: "c55ec9e2d1e14f4c002622a5cf57a90b"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.903975 10830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "b54e62f5b24844fe8385bd18754d3919"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "fdc86abbe8b35d1fe46fd5c1ef1cab9e"
server_key_iv: "c55ec9e2d1e14f4c002622a5cf57a90b"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.907812 10830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 04:48:05.910580 10860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.911880 10830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:48:05.912017 10830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "b54e62f5b24844fe8385bd18754d3919"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "fdc86abbe8b35d1fe46fd5c1ef1cab9e"
server_key_iv: "c55ec9e2d1e14f4c002622a5cf57a90b"
server_key_version: "encryptionkey@0"
I20260808 04:48:05.912133 10830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:05.932281 10830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:05.935456 10830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:05.935686 10830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:05.936327 10830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:05.937248 10830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:05.937299 10830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.937350 10830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:05.937407 10830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:05.946970 11033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:33225 every 8 connection(s)
I20260808 04:48:05.946970 10830 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:33225
I20260808 04:48:05.947989 10830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:48:05.952558 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 10830
I20260808 04:48:05.952692 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:48:05.953008 22538 external_mini_cluster.cc:1428] Setting key d7e24091c2997735ce45ffebc53681b4
Aug 08 04:48:05 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164485, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:05.962505 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.949993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35309 (local address 127.22.2.190:42233)
0808 04:48:05.950153 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:05.950157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:05.951068 (+ 911us) server_negotiation.cc:407] Connection header received
0808 04:48:05.952606 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:05.952610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:05.952676 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:05.952773 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:05.954438 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.955238 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.956563 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.956855 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.956939 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.959358 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:05.959362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:05.959364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:05.959389 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:05.960671 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.961146 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.961149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.961150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.961229 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.961558 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.961561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.961562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.961733 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:05.961803 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:05.962283 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 04:48:05.962387 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 04:48:05.963410 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.950443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:35309)
0808 04:48:05.950912 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:05.950955 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:05.952305 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:05.953032 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:05.953044 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:05.953536 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:05.954254 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.954283 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.955380 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.955384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.956415 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.956425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.956987 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.956990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.957140 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.957697 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:05.957713 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:05.959202 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:05.960790 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:05.960795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:05.960809 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:05.961057 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:05.961346 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:05.961348 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:05.961349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:05.961479 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:05.961830 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:05.961836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:05.962061 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 04:48:05.962215 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 04:48:05.962480 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":118,"threads_started":1}
I20260808 04:48:05.964597 11034 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42233
I20260808 04:48:05.964830 11034 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:05.965286 11034 heartbeater.cc:511] Master 127.22.2.190:42233 requested a full tablet report, sending...
I20260808 04:48:05.966423 10341 ts_manager.cc:194] Registered new tserver with Master: b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
I20260808 04:48:05.967101 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:48:05.967188 10341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:35309
I20260808 04:48:05.976213 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.969359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.22.2.190:42233)
0808 04:48:05.969504 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:05.969508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:05.969691 (+ 183us) server_negotiation.cc:407] Connection header received
0808 04:48:05.969814 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:05.969816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:05.969853 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:05.969920 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:05.970599 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.971082 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.971872 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.972072 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.972149 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.973527 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:05.973530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:05.973532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:05.973551 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:05.974738 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.975118 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.975120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.975121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.975192 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:05.975420 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:05.975422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:05.975423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:05.975612 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:05.975716 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:05.975926 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 04:48:05.976013 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 04:48:05.978987 10341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:48:05.982134 10341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:48:05.997561 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.992056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.22.2.129:45129)
0808 04:48:05.992794 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:05.992815 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:05.992839 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:48:05.992919 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:05.992926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:05.993110 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:05.993262 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:05.993744 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.994861 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.996314 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.996962 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.997089 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.997235 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:05.997319 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 04:48:05.997418 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":73,"threads_started":1}
I20260808 04:48:05.997550 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.991863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45129 (local address 127.0.0.1:39352)
0808 04:48:05.992594 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:05.992608 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:05.992688 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:05.993480 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:05.993485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:05.993505 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:05.993607 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.993614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.995011 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.995015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.996180 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.996188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.997093 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.997097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.997181 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.997203 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 04:48:05.997268 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 04:48:05.997400 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":34,"thread_start_us":99,"threads_started":1}
I20260808 04:48:05.997632 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.991680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.22.2.130:39707)
0808 04:48:05.992881 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:05.992892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:05.992911 (+ 19us) server_negotiation.cc:407] Connection header received
0808 04:48:05.992975 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:05.992979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:05.993119 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:05.993272 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:05.993504 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.994309 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.995855 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.996627 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.996733 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.996899 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:05.997376 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 04:48:05.997474 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1105,"thread_start_us":176,"threads_started":1}
I20260808 04:48:05.998061 11056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.991182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:39707 (local address 127.0.0.1:32854)
0808 04:48:05.992462 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:05.992475 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:05.992629 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:05.993250 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:05.993256 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:05.993280 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:05.993368 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.993373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.994457 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.994461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.995727 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.995737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.996969 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.996973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.997100 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.997144 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 04:48:05.997252 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 04:48:05.997934 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"mutex_wait_us":621,"thread_start_us":132,"threads_started":1}
I20260808 04:48:05.998329 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.991133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:33225 (local address 127.0.0.1:42478)
0808 04:48:05.992090 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:05.992145 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:05.992277 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:05.993008 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:05.993013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:05.993033 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:05.993220 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.993232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.994661 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.994664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.996040 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:05.996052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.997336 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.997341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:05.997424 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.997444 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 04:48:05.997513 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 04:48:05.997577 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":76,"thread_start_us":85,"threads_started":1}
I20260808 04:48:05.998654 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:05.991270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42478 (local address 127.22.2.131:33225)
0808 04:48:05.992270 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:05.992286 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:05.992326 (+ 40us) server_negotiation.cc:407] Connection header received
0808 04:48:05.992410 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:05.992418 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:05.992673 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:05.992828 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:05.993371 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.994508 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.996185 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:05.997020 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:05.997120 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:05.997270 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:05.998008 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 04:48:05.998108 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":133,"threads_started":1}
I20260808 04:48:06.000813 10522 tablet_service.cc:1467] Processing CreateTablet for tablet 36414dca0d1b476da3fbe779bec97d2f (DEFAULT_TABLE table=test-table [id=9476dcdb8f6d4b21bae2cdcf414050de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:06.000998 10724 tablet_service.cc:1467] Processing CreateTablet for tablet 36414dca0d1b476da3fbe779bec97d2f (DEFAULT_TABLE table=test-table [id=9476dcdb8f6d4b21bae2cdcf414050de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:06.001403 10926 tablet_service.cc:1467] Processing CreateTablet for tablet 36414dca0d1b476da3fbe779bec97d2f (DEFAULT_TABLE table=test-table [id=9476dcdb8f6d4b21bae2cdcf414050de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:06.001969 10522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36414dca0d1b476da3fbe779bec97d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:06.002065 10724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36414dca0d1b476da3fbe779bec97d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:06.002354 10926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36414dca0d1b476da3fbe779bec97d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:06.008579 11058 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Bootstrap starting.
I20260808 04:48:06.010493 11059 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Bootstrap starting.
I20260808 04:48:06.010840 11058 tablet_bootstrap.cc:654] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:06.011591 11058 log.cc:824] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:06.012921 11060 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Bootstrap starting.
I20260808 04:48:06.013005 11059 tablet_bootstrap.cc:654] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:06.013897 11059 log.cc:824] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:06.014047 11058 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: No bootstrap required, opened a new log
I20260808 04:48:06.014259 11058 ts_tablet_manager.cc:1397] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:48:06.014808 11060 tablet_bootstrap.cc:654] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:06.015942 11060 log.cc:824] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:06.016122 11059 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: No bootstrap required, opened a new log
I20260808 04:48:06.016319 11059 ts_tablet_manager.cc:1397] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260808 04:48:06.017764 11060 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: No bootstrap required, opened a new log
I20260808 04:48:06.017971 11060 ts_tablet_manager.cc:1397] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:48:06.018790 11058 raft_consensus.cc:348] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.019191 11058 raft_consensus.cc:374] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:06.019294 11058 raft_consensus.cc:729] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e21ba4f65ec48c0a229ff1cd8fc8b84, State: Initialized, Role: FOLLOWER
I20260808 04:48:06.019855 11058 consensus_queue.cc:260] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.020298 11059 raft_consensus.cc:348] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.020573 11059 raft_consensus.cc:374] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:06.020633 11059 raft_consensus.cc:729] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 395be1bb74ad4137a45158f22ce64607, State: Initialized, Role: FOLLOWER
I20260808 04:48:06.020648 10614 heartbeater.cc:503] Master 127.22.2.190:42233 was elected leader, sending a full tablet report...
I20260808 04:48:06.020979 11058 ts_tablet_manager.cc:1428] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 04:48:06.021098 11059 consensus_queue.cc:260] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.021740 10821 heartbeater.cc:503] Master 127.22.2.190:42233 was elected leader, sending a full tablet report...
I20260808 04:48:06.022444 11060 raft_consensus.cc:348] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.022662 11060 raft_consensus.cc:374] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:06.022735 11060 raft_consensus.cc:729] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54e62f5b24844fe8385bd18754d3919, State: Initialized, Role: FOLLOWER
I20260808 04:48:06.023166 11060 consensus_queue.cc:260] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.023815 11034 heartbeater.cc:503] Master 127.22.2.190:42233 was elected leader, sending a full tablet report...
I20260808 04:48:06.024160 11060 ts_tablet_manager.cc:1428] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:48:06.025110 11059 ts_tablet_manager.cc:1428] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
W20260808 04:48:06.199126 11035 tablet.cc:2369] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:48:06.220026 10822 tablet.cc:2369] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:48:06.241158 10615 tablet.cc:2369] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:48:06.351269 11067 raft_consensus.cc:482] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:06.351513 11067 raft_consensus.cc:504] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.352749 11067 leader_election.cc:288] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129:45129)
I20260808 04:48:06.356806 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.352989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:39707 (local address 127.22.2.131:35881)
0808 04:48:06.353168 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:06.353186 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:06.353289 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:06.353655 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:06.353658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:06.353676 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:06.353797 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:06.353803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.354648 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.354652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:06.355670 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:06.355681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.356311 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.356315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:06.356424 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.356446 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 04:48:06.356590 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 04:48:06.356667 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":29}
I20260808 04:48:06.356824 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.353238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35881 (local address 127.22.2.130:39707)
0808 04:48:06.353416 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:06.353420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:06.353431 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:48:06.353506 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:06.353509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:06.353558 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:06.353639 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:06.353914 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.354453 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.355795 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.356187 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.356316 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.356370 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:06.356612 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 04:48:06.356704 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 04:48:06.357077 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.353414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:51449 (local address 127.22.2.129:45129)
0808 04:48:06.353570 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:06.353576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:06.353749 (+ 173us) server_negotiation.cc:407] Connection header received
0808 04:48:06.353910 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:06.353914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:06.353964 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:06.354049 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:06.354324 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.354889 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.356082 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.356470 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.356654 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.356702 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:06.356902 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 04:48:06.356966 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 04:48:06.357092 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.353290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45129 (local address 127.22.2.131:51449)
0808 04:48:06.353654 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:06.353666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:06.353790 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:06.354058 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:06.354061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:06.354076 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:06.354202 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:06.354208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.355002 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.355007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:06.355943 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:06.355953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.356646 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.356649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:06.356747 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.356774 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 04:48:06.356885 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 04:48:06.356951 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":2,"thread_start_us":129,"threads_started":1}
I20260808 04:48:06.357632 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36414dca0d1b476da3fbe779bec97d2f" candidate_uuid: "b54e62f5b24844fe8385bd18754d3919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395be1bb74ad4137a45158f22ce64607" is_pre_election: true
I20260808 04:48:06.357674 10551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36414dca0d1b476da3fbe779bec97d2f" candidate_uuid: "b54e62f5b24844fe8385bd18754d3919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" is_pre_election: true
I20260808 04:48:06.357940 10758 raft_consensus.cc:2454] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54e62f5b24844fe8385bd18754d3919 in term 0.
I20260808 04:48:06.357973 10551 raft_consensus.cc:2454] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54e62f5b24844fe8385bd18754d3919 in term 0.
I20260808 04:48:06.358420 10864 leader_election.cc:302] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e21ba4f65ec48c0a229ff1cd8fc8b84, b54e62f5b24844fe8385bd18754d3919; no voters:
I20260808 04:48:06.358705 11067 raft_consensus.cc:2781] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:48:06.358799 11067 raft_consensus.cc:482] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:48:06.358847 11067 raft_consensus.cc:3037] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:06.359925 11067 raft_consensus.cc:504] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.360263 11067 leader_election.cc:288] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [CANDIDATE]: Term 1 election: Requested vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129:45129)
I20260808 04:48:06.360677 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36414dca0d1b476da3fbe779bec97d2f" candidate_uuid: "b54e62f5b24844fe8385bd18754d3919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395be1bb74ad4137a45158f22ce64607"
I20260808 04:48:06.360729 10551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36414dca0d1b476da3fbe779bec97d2f" candidate_uuid: "b54e62f5b24844fe8385bd18754d3919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84"
I20260808 04:48:06.360800 10758 raft_consensus.cc:3037] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:06.360893 10551 raft_consensus.cc:3037] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:06.361821 10758 raft_consensus.cc:2454] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54e62f5b24844fe8385bd18754d3919 in term 1.
I20260808 04:48:06.362016 10551 raft_consensus.cc:2454] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54e62f5b24844fe8385bd18754d3919 in term 1.
I20260808 04:48:06.362118 10864 leader_election.cc:302] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 395be1bb74ad4137a45158f22ce64607, b54e62f5b24844fe8385bd18754d3919; no voters:
I20260808 04:48:06.362401 11067 raft_consensus.cc:2781] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:48:06.362581 11067 raft_consensus.cc:686] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 1 LEADER]: Becoming Leader. State: Replica: b54e62f5b24844fe8385bd18754d3919, State: Running, Role: LEADER
I20260808 04:48:06.362902 11067 consensus_queue.cc:237] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.366128 10339 catalog_manager.cc:5495] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 reported cstate change: term changed from 0 to 1, leader changed from <none> to b54e62f5b24844fe8385bd18754d3919 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "b54e62f5b24844fe8385bd18754d3919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } health_report { overall_health: UNKNOWN } } }
I20260808 04:48:06.447229 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.443533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42484 (local address 127.22.2.131:33225)
0808 04:48:06.443695 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:06.443700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:06.443717 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:48:06.443822 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:06.443825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:06.443883 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:06.443997 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:48:06.444272 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.444841 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.445708 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.445930 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.446047 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.446391 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:48:06.446809 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:48:06.446948 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:06.447010 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 04:48:06.447066 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 04:48:06.453186 10551 raft_consensus.cc:1261] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Refusing update from remote peer b54e62f5b24844fe8385bd18754d3919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:06.453186 10758 raft_consensus.cc:1261] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Refusing update from remote peer b54e62f5b24844fe8385bd18754d3919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:06.453986 11067 consensus_queue.cc:1035] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:06.454245 11069 consensus_queue.cc:1035] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:06.462245 11075 mvcc.cc:205] Tried to move back new op lower bound from 7316129736504586240 to 7316129736144728064. Current Snapshot: MvccSnapshot[applied={T|T < 7316129736504586240}]
I20260808 04:48:06.472297 10339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:48:06.472438 10339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:48:06.474962 10339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Sending DeleteTablet for 3 replicas of tablet 36414dca0d1b476da3fbe779bec97d2f
I20260808 04:48:06.475728 10926 tablet_service.cc:1514] Processing DeleteTablet for tablet 36414dca0d1b476da3fbe779bec97d2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:06 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:42478
I20260808 04:48:06.475751 10522 tablet_service.cc:1514] Processing DeleteTablet for tablet 36414dca0d1b476da3fbe779bec97d2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:06 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39352
I20260808 04:48:06.475808 10724 tablet_service.cc:1514] Processing DeleteTablet for tablet 36414dca0d1b476da3fbe779bec97d2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:06 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:32854
I20260808 04:48:06.476262 11089 tablet_replica.cc:330] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: stopping tablet replica
I20260808 04:48:06.476341 11090 tablet_replica.cc:330] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: stopping tablet replica
I20260808 04:48:06.476436 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 10428
I20260808 04:48:06.476629 11089 raft_consensus.cc:2229] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:06.476676 11090 raft_consensus.cc:2229] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:06.477130 11088 tablet_replica.cc:330] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: stopping tablet replica
I20260808 04:48:06.477468 11088 raft_consensus.cc:2229] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:48:06.477942 11088 raft_consensus.cc:2258] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:06.480507 11090 raft_consensus.cc:2258] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:06.483850 11088 ts_tablet_manager.cc:1905] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:06.485853 11090 ts_tablet_manager.cc:1905] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:06.487004 11088 ts_tablet_manager.cc:1918] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:06.487197 11088 log.cc:1196] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/36414dca0d1b476da3fbe779bec97d2f
I20260808 04:48:06.487774 11088 ts_tablet_manager.cc:1939] T 36414dca0d1b476da3fbe779bec97d2f P b54e62f5b24844fe8385bd18754d3919: Deleting consensus metadata
I20260808 04:48:06.488878 11090 ts_tablet_manager.cc:1918] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:06.489063 11090 log.cc:1196] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/36414dca0d1b476da3fbe779bec97d2f
I20260808 04:48:06.489485 11090 ts_tablet_manager.cc:1939] T 36414dca0d1b476da3fbe779bec97d2f P 395be1bb74ad4137a45158f22ce64607: Deleting consensus metadata
I20260808 04:48:06.489563 10320 catalog_manager.cc:4841] TS b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225): tablet 36414dca0d1b476da3fbe779bec97d2f (table test-table [id=9476dcdb8f6d4b21bae2cdcf414050de]) successfully deleted
I20260808 04:48:06.490624 10319 catalog_manager.cc:4841] TS 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707): tablet 36414dca0d1b476da3fbe779bec97d2f (table test-table [id=9476dcdb8f6d4b21bae2cdcf414050de]) successfully deleted
W20260808 04:48:06.493892 10319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:45129 (error 108)
I20260808 04:48:06.494136 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 10626
I20260808 04:48:06.496887 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.494596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45129 (local address 127.0.0.1:39368)
0808 04:48:06.496702 (+ 2106us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:06.496787 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:45129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1897}
W20260808 04:48:06.498396 10319 catalog_manager.cc:4584] TS 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129:45129): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36414dca0d1b476da3fbe779bec97d2f: Network error: Client connection negotiation failed: client connection to 127.22.2.129:45129: connect: Connection refused (error 111)
I20260808 04:48:06.505348 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 10830
I20260808 04:48:06.521701 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 10296
I20260808 04:48:06.536940 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42233
--webserver_interface=127.22.2.190
--webserver_port=35517
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:06.649950 11092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:06.650231 11092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:06.650312 11092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:06.654307 11092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:06.654385 11092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:06.654423 11092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:06.654481 11092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:06.654522 11092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:06.659592 11092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=35517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:06.660683 11092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:06.661753 11092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:06.675603 11100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:06.675591 11101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:06.675591 11103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:06.676010 11092 server_base.cc:1034] running on GCE node
I20260808 04:48:06.676633 11092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:06.677772 11092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:06.678967 11092 hybrid_clock.cc:648] HybridClock initialized: now 1786164486678938 us; error 48 us; skew 500 ppm
Aug 08 04:48:06 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164486, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:06.681941 11092 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:48:06.682965 11092 webserver.cc:460] Webserver started at http://127.22.2.190:35517/ using document root <none> and password file <none>
I20260808 04:48:06.683516 11092 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:06.683586 11092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:06.687570 11092 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:48:06.689867 11115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:06.691089 11092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:48:06.691193 11092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "e54513a6d81e4608b09252c2b95ec61b"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "c98349ddc74df5d1e8337b059b0161c6"
server_key_iv: "fd8f1b3b9dfb5e92251ed41c862cdfa4"
server_key_version: "encryptionkey@0"
I20260808 04:48:06.691749 11092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:06.717367 11092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:06.720861 11092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:06.721031 11092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:06.728574 11092 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42233
I20260808 04:48:06.728909 11190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42233 every 8 connection(s)
I20260808 04:48:06.729773 11092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:48:06.733634 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 11092
I20260808 04:48:06.734081 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:45129
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:42233
--webserver_port=36241
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:06.734606 11192 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:48:06.736310 11192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Bootstrap starting.
I20260808 04:48:06.754253 11192 log.cc:824] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:06.763278 11192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:48:06.763664 11192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Bootstrap complete.
I20260808 04:48:06.767951 11192 raft_consensus.cc:348] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:06.768440 11192 raft_consensus.cc:729] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54513a6d81e4608b09252c2b95ec61b, State: Initialized, Role: FOLLOWER
I20260808 04:48:06.769009 11192 consensus_queue.cc:260] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:06.769196 11192 raft_consensus.cc:388] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:48:06.769287 11192 raft_consensus.cc:482] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:48:06.769408 11192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:48:06.771287 11192 raft_consensus.cc:504] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:06.771638 11192 leader_election.cc:302] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e54513a6d81e4608b09252c2b95ec61b; no voters:
I20260808 04:48:06.771953 11192 leader_election.cc:288] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 04:48:06.772079 11199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 04:48:06.772336 11199 raft_consensus.cc:686] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 2 LEADER]: Becoming Leader. State: Replica: e54513a6d81e4608b09252c2b95ec61b, State: Running, Role: LEADER
I20260808 04:48:06.772691 11199 consensus_queue.cc:237] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:06.773249 11192 sys_catalog.cc:564] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:06.773665 11199 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e54513a6d81e4608b09252c2b95ec61b. Latest consensus state: current_term: 2 leader_uuid: "e54513a6d81e4608b09252c2b95ec61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } } }
I20260808 04:48:06.773792 11199 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:06.774163 11207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:48:06.774269 11202 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e54513a6d81e4608b09252c2b95ec61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } } }
I20260808 04:48:06.774461 11202 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:06.778997 11207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:48:06.779825 11207 catalog_manager.cc:1259] Loaded cluster ID: b07eea6058584cf5a0de91d0029c5b94
I20260808 04:48:06.779888 11207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:48:06.781502 11207 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 04:48:06.786077 11207 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 04:48:06.791846 11207 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:48:06.792748 11207 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Loaded TSK: 0
I20260808 04:48:06.794015 11207 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 04:48:06.862207 11196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:06.862461 11196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:06.862505 11196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:06.866511 11196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:06.866565 11196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:06.866623 11196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:48:06.872006 11196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:45129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=36241
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:06.873102 11196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:06.874235 11196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:06.885387 11241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:06.885419 11236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:06.885368 11239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:06.885854 11196 server_base.cc:1034] running on GCE node
I20260808 04:48:06.886292 11196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:06.887009 11196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:06.888197 11196 hybrid_clock.cc:648] HybridClock initialized: now 1786164486888179 us; error 41 us; skew 500 ppm
Aug 08 04:48:06 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164486, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:06.891167 11196 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:48:06.892159 11196 webserver.cc:460] Webserver started at http://127.22.2.129:36241/ using document root <none> and password file <none>
I20260808 04:48:06.892804 11196 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:06.892886 11196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:06.897140 11196 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:48:06.899397 11251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:06.900749 11196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:48:06.900871 11196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "ccd20a855e0e3100df1b3084988ee93a"
server_key_iv: "b4a86bc8ccfee5e30eced0cf77b390f5"
server_key_version: "encryptionkey@0"
I20260808 04:48:06.901470 11196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:06.920802 11196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:06.924252 11196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:06.924481 11196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:06.925418 11196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:06.926663 11258 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 04:48:06.931097 11196 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 04:48:06.931233 11196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 04:48:06.931349 11196 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 04:48:06.933275 11196 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 04:48:06.933326 11196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 04:48:06.933756 11258 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Bootstrap starting.
I20260808 04:48:06.945436 11196 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:45129
I20260808 04:48:06.945528 11406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:45129 every 8 connection(s)
I20260808 04:48:06.946614 11196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:48:06.952097 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 11196
I20260808 04:48:06.952673 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:39707
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:42233
--webserver_port=37487
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:06.953212 11258 log.cc:824] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:48:06 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164486, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:06.962939 11258 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:48:06.963462 11258 tablet_bootstrap.cc:492] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Bootstrap complete.
I20260808 04:48:06.965073 11258 ts_tablet_manager.cc:1397] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.012s
I20260808 04:48:06.966236 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.949376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:48389)
0808 04:48:06.950007 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:06.950066 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:06.951435 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:06.953509 (+ 2074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:06.953522 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:06.954113 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:06.954870 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:06.954889 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.956316 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.956320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:06.957348 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:06.957360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.958162 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.958167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:06.958385 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.959057 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:06.959076 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:06.960714 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:06.963152 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:06.963156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:06.963167 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:06.963422 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:06.963812 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:06.963815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:06.963817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:06.963949 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:06.964550 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:06.964557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:06.964804 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 04:48:06.965000 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 04:48:06.965330 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":170,"threads_started":1}
I20260808 04:48:06.966223 11416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.949540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48389 (local address 127.22.2.190:42233)
0808 04:48:06.950018 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:06.950038 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:06.950200 (+ 162us) server_negotiation.cc:407] Connection header received
0808 04:48:06.952022 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:06.952040 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:06.952505 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:06.952955 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:06.955075 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.956136 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.957494 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.958002 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.958223 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.960923 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:06.960929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:06.960942 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:06.960972 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:06.962972 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:06.963552 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:06.963558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:06.963564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:06.963691 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:06.964053 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:06.964058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:06.964060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:06.964408 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:06.964592 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:06.965120 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 04:48:06.965398 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":115,"threads_started":1}
I20260808 04:48:06.968642 11258 raft_consensus.cc:348] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.969218 11258 raft_consensus.cc:729] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e21ba4f65ec48c0a229ff1cd8fc8b84, State: Initialized, Role: FOLLOWER
I20260808 04:48:06.969395 11407 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42233
I20260808 04:48:06.969635 11407 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:06.969750 11258 consensus_queue.cc:260] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:06.970252 11407 heartbeater.cc:511] Master 127.22.2.190:42233 requested a full tablet report, sending...
I20260808 04:48:06.970778 11258 ts_tablet_manager.cc:1428] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:48:06.972815 11139 ts_manager.cc:194] Registered new tserver with Master: 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129:45129)
I20260808 04:48:06.976269 11139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:48389
I20260808 04:48:06.978997 11407 heartbeater.cc:503] Master 127.22.2.190:42233 was elected leader, sending a full tablet report...
Aug 08 04:48:06 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164486, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:48:06.986514 11422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.975702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45129 (local address 127.0.0.1:39374)
0808 04:48:06.976500 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:06.976572 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:06.976712 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:06.977407 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:06.977412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:06.977737 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:06.977836 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:06.977842 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.978861 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.978866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:06.979647 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:06.979660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.980537 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.980542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:06.980643 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.981238 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:06.981250 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:06.982625 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:06.984806 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:06.984837 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:06.984844 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:06.985117 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:06.985436 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:06.985438 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:06.985440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:06.985517 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:06.986003 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:06.986008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:06.986107 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 04:48:06.986218 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 04:48:06.986383 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":102,"threads_started":1}
I20260808 04:48:06.986652 11424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:06.975815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.22.2.129:45129)
0808 04:48:06.976158 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:06.976169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:06.976672 (+ 503us) server_negotiation.cc:407] Connection header received
0808 04:48:06.976861 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:06.976866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:06.977070 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:06.977236 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:06.977962 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.978737 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.979773 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:06.980281 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:06.980408 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:06.982768 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:06.982779 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:06.982792 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:06.982820 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:06.984700 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:06.985235 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:06.985241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:06.985243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:06.985339 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:06.985604 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:06.985607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:06.985609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:06.985888 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:06.986023 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:06.986328 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 04:48:06.986490 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":262,"thread_start_us":107,"threads_started":1}
I20260808 04:48:06.987784 11323 tablet_service.cc:1514] Processing DeleteTablet for tablet 36414dca0d1b476da3fbe779bec97d2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:06 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39374
I20260808 04:48:06.988693 11425 tablet_replica.cc:330] stopping tablet replica
I20260808 04:48:06.989022 11425 raft_consensus.cc:2229] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:06.989336 11425 raft_consensus.cc:2258] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:06.990343 11425 ts_tablet_manager.cc:1905] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:06.993885 11425 ts_tablet_manager.cc:1918] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:06.993969 11425 log.cc:1196] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/36414dca0d1b476da3fbe779bec97d2f
I20260808 04:48:06.994230 11425 ts_tablet_manager.cc:1939] T 36414dca0d1b476da3fbe779bec97d2f P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Deleting consensus metadata
I20260808 04:48:06.996042 11118 catalog_manager.cc:4841] TS 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129:45129): tablet 36414dca0d1b476da3fbe779bec97d2f (table test-table [id=9476dcdb8f6d4b21bae2cdcf414050de]) successfully deleted
W20260808 04:48:07.076159 11419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:07.076421 11419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:07.076488 11419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:07.080361 11419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:07.080454 11419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:07.080605 11419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:48:07.085772 11419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:39707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=37487
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:07.086817 11419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:07.087812 11419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:07.097589 11439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:07.097579 11433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:07.097594 11431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:07.097824 11419 server_base.cc:1034] running on GCE node
I20260808 04:48:07.098284 11419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:07.098948 11419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:07.100147 11419 hybrid_clock.cc:648] HybridClock initialized: now 1786164487100136 us; error 28 us; skew 500 ppm
Aug 08 04:48:07 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164487, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:07.103040 11419 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:48:07.103989 11419 webserver.cc:460] Webserver started at http://127.22.2.130:37487/ using document root <none> and password file <none>
I20260808 04:48:07.104653 11419 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:07.104790 11419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:07.108990 11419 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:48:07.111289 11447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:07.112700 11419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:48:07.112849 11419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "395be1bb74ad4137a45158f22ce64607"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "dcc2a36559708428a9388f201ba5b63f"
server_key_iv: "890c6b09a840e99e537f052df92b9a96"
server_key_version: "encryptionkey@0"
I20260808 04:48:07.113420 11419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:07.129598 11419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:07.132620 11419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:07.132836 11419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:07.133486 11419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:07.134394 11419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:07.134474 11419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:07.134546 11419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:07.134600 11419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:07.144688 11419 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:39707
I20260808 04:48:07.144709 11615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:39707 every 8 connection(s)
I20260808 04:48:07.145815 11419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:48:07.151207 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 11419
I20260808 04:48:07.151678 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:33225
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:42233
--webserver_port=46841
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:48:07 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164487, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:07.161839 11416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.147748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:37685 (local address 127.22.2.190:42233)
0808 04:48:07.147919 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.147923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.148905 (+ 982us) server_negotiation.cc:407] Connection header received
0808 04:48:07.150227 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.150230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.150283 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.150366 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:07.151848 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.152468 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.154096 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.154556 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.154693 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.157735 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:07.157739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:07.157741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:07.157767 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:07.159329 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:07.160026 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:07.160028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:07.160029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:07.160107 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:07.160606 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:07.160609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:07.160610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:07.160782 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:07.160872 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.161595 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.161701 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 04:48:07.163196 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.148116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:37685)
0808 04:48:07.148685 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.148751 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:07.150022 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:07.150534 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:07.150540 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:07.150910 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:07.151661 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.151682 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.152688 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.152694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.153958 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.153968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.154713 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.154716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.154849 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.155469 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:07.155485 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:07.157545 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:07.159464 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:07.159473 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:07.159493 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:07.159914 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:07.160221 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:07.160228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:07.160232 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:07.160448 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:07.160917 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:07.160926 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:07.161305 (+ 379us) client_negotiation.cc:769] Sending connection context
0808 04:48:07.161547 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 04:48:07.161950 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":130,"threads_started":1}
I20260808 04:48:07.164604 11616 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42233
I20260808 04:48:07.164858 11616 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:07.165364 11616 heartbeater.cc:511] Master 127.22.2.190:42233 requested a full tablet report, sending...
I20260808 04:48:07.166443 11139 ts_manager.cc:194] Registered new tserver with Master: 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707)
I20260808 04:48:07.167098 11139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:37685
W20260808 04:48:07.274582 11619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:07.274879 11619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:07.274963 11619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:07.279130 11619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:07.279217 11619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:07.279330 11619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:48:07.284619 11619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:33225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=46841
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:07.285646 11619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:07.286645 11619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:07.297179 11632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:07.297222 11630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:07.297235 11629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:07.297858 11619 server_base.cc:1034] running on GCE node
I20260808 04:48:07.298269 11619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:07.298936 11619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:07.300129 11619 hybrid_clock.cc:648] HybridClock initialized: now 1786164487300110 us; error 33 us; skew 500 ppm
Aug 08 04:48:07 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164487, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:07.302896 11619 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:48:07.303853 11619 webserver.cc:460] Webserver started at http://127.22.2.131:46841/ using document root <none> and password file <none>
I20260808 04:48:07.304620 11619 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:07.304714 11619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:07.308713 11619 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:48:07.310789 11645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:07.312018 11619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:48:07.312143 11619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "b54e62f5b24844fe8385bd18754d3919"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "fdc86abbe8b35d1fe46fd5c1ef1cab9e"
server_key_iv: "c55ec9e2d1e14f4c002622a5cf57a90b"
server_key_version: "encryptionkey@0"
I20260808 04:48:07.312754 11619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:07.354436 11619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:07.357698 11619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:07.357914 11619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:07.358577 11619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:07.359422 11619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:07.359501 11619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:07.359565 11619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:07.359601 11619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:07.370021 11619 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:33225
I20260808 04:48:07.370054 11826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:33225 every 8 connection(s)
I20260808 04:48:07.371124 11619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:48:07.380178 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 11619
Aug 08 04:48:07 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164487, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:07.386749 11416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.373444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:33111 (local address 127.22.2.190:42233)
0808 04:48:07.373612 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.373618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.374749 (+ 1131us) server_negotiation.cc:407] Connection header received
0808 04:48:07.376200 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.376203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.376256 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.376343 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:07.377522 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.378116 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.379257 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.379497 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.379603 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.382469 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:07.382473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:07.382476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:07.382502 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:07.384140 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:07.384809 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:07.384813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:07.384814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:07.384933 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:07.385319 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:07.385323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:07.385324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:07.385565 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:07.385673 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.386394 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.386548 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 04:48:07.387606 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.373901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:33111)
0808 04:48:07.374559 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.374629 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:07.375932 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:07.376512 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:07.376519 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:07.376881 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:07.377374 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.377389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.378235 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.378237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.379104 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.379113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.379644 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.379647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.379786 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.380466 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:07.380482 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:07.382293 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:07.384273 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:07.384279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:07.384299 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:07.384696 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:07.385044 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:07.385047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:07.385049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:07.385191 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:07.385737 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:07.385746 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:07.386086 (+ 340us) client_negotiation.cc:769] Sending connection context
0808 04:48:07.386322 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 04:48:07.386619 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":287,"threads_started":1}
I20260808 04:48:07.388880 11827 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42233
I20260808 04:48:07.389137 11827 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:07.389645 11827 heartbeater.cc:511] Master 127.22.2.190:42233 requested a full tablet report, sending...
I20260808 04:48:07.389900 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.381012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.22.2.190:42233)
0808 04:48:07.381305 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.381310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.381412 (+ 102us) server_negotiation.cc:407] Connection header received
0808 04:48:07.381519 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.381524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.381585 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.381692 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:07.382673 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.383445 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.384435 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.384829 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.384918 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.386476 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:07.386479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:07.386481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:07.386508 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:07.388151 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:07.388620 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:07.388622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:07.388624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:07.388699 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:07.388984 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:07.388987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:07.388989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:07.389155 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:07.389381 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.389597 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.389718 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":97,"threads_started":1}
I20260808 04:48:07.390822 11139 ts_manager.cc:194] Registered new tserver with Master: b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
I20260808 04:48:07.391368 11139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:33111
I20260808 04:48:07.391809 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:48:07.396129 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.392413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.22.2.190:42233)
0808 04:48:07.392521 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.392542 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.392834 (+ 292us) server_negotiation.cc:407] Connection header received
0808 04:48:07.392954 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.392956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.393036 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.393113 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:48:07.393404 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.393893 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.394795 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.395027 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.395142 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.395446 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:48:07.395787 (+ 341us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:48:07.395919 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.395964 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.396016 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 04:48:07.396847 11139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:48:07.400007 11139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:48:07.408085 11323 tablet_service.cc:1467] Processing CreateTablet for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0 (DEFAULT_TABLE table=test-table [id=23f28acda6ff49db8d07869477cc5657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:07.409075 11323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:07.412017 11422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.406944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:39707 (local address 127.0.0.1:32864)
0808 04:48:07.407057 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.407074 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:07.407157 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:07.408162 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:07.408165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:07.408184 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:07.408276 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.408282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.409631 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.409637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.410512 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.410522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.411539 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.411543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.411652 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.411706 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 04:48:07.411802 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 04:48:07.411889 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":47}
I20260808 04:48:07.412115 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.407126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.22.2.130:39707)
0808 04:48:07.407646 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.407659 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.407682 (+ 23us) server_negotiation.cc:407] Connection header received
0808 04:48:07.407763 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.407769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.407995 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.408176 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:07.408363 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.409472 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.410646 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.411399 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.411511 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.411661 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.411850 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.411968 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":118,"threads_started":1}
I20260808 04:48:07.412235 11258 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Bootstrap starting.
I20260808 04:48:07.412397 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.407231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:33225 (local address 127.0.0.1:42498)
0808 04:48:07.407473 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.407486 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:07.407573 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:07.408455 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:07.408458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:07.408475 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:07.408583 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.408590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.409713 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.409717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.410865 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.410878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.411995 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.411999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.412100 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.412128 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 04:48:07.412185 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 04:48:07.412264 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":111,"threads_started":1}
I20260808 04:48:07.413187 11258 tablet_bootstrap.cc:654] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:07.413455 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.407657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.22.2.131:33225)
0808 04:48:07.408003 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.408013 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.408030 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:48:07.408089 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.408115 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.408322 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.408473 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:07.408680 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.409546 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.411006 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.411843 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.413080 (+ 1237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.413190 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.413259 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.413345 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":162,"threads_started":1}
I20260808 04:48:07.414752 11258 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: No bootstrap required, opened a new log
I20260808 04:48:07.414850 11258 ts_tablet_manager.cc:1397] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:48:07.415335 11258 raft_consensus.cc:348] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.415464 11258 raft_consensus.cc:374] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:07.415504 11258 raft_consensus.cc:729] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e21ba4f65ec48c0a229ff1cd8fc8b84, State: Initialized, Role: FOLLOWER
I20260808 04:48:07.415638 11258 consensus_queue.cc:260] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.416388 11258 ts_tablet_manager.cc:1428] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:48:07.416769 11726 tablet_service.cc:1467] Processing CreateTablet for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0 (DEFAULT_TABLE table=test-table [id=23f28acda6ff49db8d07869477cc5657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:07.417191 11515 tablet_service.cc:1467] Processing CreateTablet for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0 (DEFAULT_TABLE table=test-table [id=23f28acda6ff49db8d07869477cc5657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:07.417992 11726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:07.418147 11515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:07.425055 11849 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Bootstrap starting.
I20260808 04:48:07.426091 11851 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Bootstrap starting.
I20260808 04:48:07.428098 11849 tablet_bootstrap.cc:654] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:07.428296 11851 tablet_bootstrap.cc:654] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:07.429119 11849 log.cc:824] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:07.429638 11851 log.cc:824] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:07.431285 11849 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: No bootstrap required, opened a new log
I20260808 04:48:07.431496 11849 ts_tablet_manager.cc:1397] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 04:48:07.431680 11851 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: No bootstrap required, opened a new log
I20260808 04:48:07.431874 11851 ts_tablet_manager.cc:1397] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:48:07.435539 11849 raft_consensus.cc:348] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.435539 11851 raft_consensus.cc:348] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.435817 11851 raft_consensus.cc:374] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:07.435817 11849 raft_consensus.cc:374] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:07.435870 11851 raft_consensus.cc:729] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54e62f5b24844fe8385bd18754d3919, State: Initialized, Role: FOLLOWER
I20260808 04:48:07.435870 11849 raft_consensus.cc:729] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 395be1bb74ad4137a45158f22ce64607, State: Initialized, Role: FOLLOWER
I20260808 04:48:07.436352 11849 consensus_queue.cc:260] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.436352 11851 consensus_queue.cc:260] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.437083 11616 heartbeater.cc:503] Master 127.22.2.190:42233 was elected leader, sending a full tablet report...
I20260808 04:48:07.437350 11827 heartbeater.cc:503] Master 127.22.2.190:42233 was elected leader, sending a full tablet report...
I20260808 04:48:07.437412 11851 ts_tablet_manager.cc:1428] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 04:48:07.437412 11849 ts_tablet_manager.cc:1428] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 04:48:07.447921 11412 tablet.cc:2369] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:48:07.622409 11828 tablet.cc:2369] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:48:07.647119 11617 tablet.cc:2369] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:48:07.789135 11420 raft_consensus.cc:482] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:07.789436 11420 raft_consensus.cc:504] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.790660 11420 leader_election.cc:288] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
I20260808 04:48:07.796324 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.791100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:43583 (local address 127.22.2.131:33225)
0808 04:48:07.791253 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.791259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.791535 (+ 276us) server_negotiation.cc:407] Connection header received
0808 04:48:07.791613 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.791616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.791666 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.791760 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:07.792232 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.793063 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.794882 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.795495 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.795651 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.795711 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.796131 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.796212 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 04:48:07.796331 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.790954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:39707 (local address 127.22.2.129:48481)
0808 04:48:07.791276 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.791294 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:07.791418 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:07.791801 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:07.791804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:07.791849 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:07.792087 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.792095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.793210 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.793214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.794727 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.794734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.795814 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.795817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.795939 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.795979 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 04:48:07.796045 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 04:48:07.796161 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":285,"spinlock_wait_cycles":30464,"thread_start_us":114,"threads_started":1}
I20260808 04:48:07.796380 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.791066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48481 (local address 127.22.2.130:39707)
0808 04:48:07.791338 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.791341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.791356 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:48:07.791604 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.791609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.791671 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.791761 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:07.792232 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.793063 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.794882 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.795572 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.795677 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.795721 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.796101 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.796238 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260808 04:48:07.796640 11856 raft_consensus.cc:482] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:07.796841 11856 raft_consensus.cc:504] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.797037 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.790951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:33225 (local address 127.22.2.129:43583)
0808 04:48:07.791393 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.791406 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:07.791496 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:07.791801 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:07.791804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:07.791851 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:07.792090 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.792095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.793210 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.793214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.794716 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.794729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.795769 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.795773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.795924 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.795977 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 04:48:07.796089 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 04:48:07.796178 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":234,"spinlock_wait_cycles":45312,"thread_start_us":88,"threads_started":1}
I20260808 04:48:07.797004 11548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b5fa2c8eb943188c5ab1c2a43c0ff0" candidate_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395be1bb74ad4137a45158f22ce64607" is_pre_election: true
I20260808 04:48:07.797289 11548 raft_consensus.cc:2454] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e21ba4f65ec48c0a229ff1cd8fc8b84 in term 0.
I20260808 04:48:07.797722 11753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b5fa2c8eb943188c5ab1c2a43c0ff0" candidate_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54e62f5b24844fe8385bd18754d3919" is_pre_election: true
I20260808 04:48:07.797796 11254 leader_election.cc:302] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 395be1bb74ad4137a45158f22ce64607, 3e21ba4f65ec48c0a229ff1cd8fc8b84; no voters:
I20260808 04:48:07.798061 11753 raft_consensus.cc:2454] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e21ba4f65ec48c0a229ff1cd8fc8b84 in term 0.
I20260808 04:48:07.798076 11420 raft_consensus.cc:2781] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:48:07.798166 11420 raft_consensus.cc:482] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:48:07.798223 11420 raft_consensus.cc:3037] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:07.799580 11420 raft_consensus.cc:504] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.799659 11856 leader_election.cc:288] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225), 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129:45129)
I20260808 04:48:07.799944 11420 leader_election.cc:288] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 1 election: Requested vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
I20260808 04:48:07.800374 11548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b5fa2c8eb943188c5ab1c2a43c0ff0" candidate_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395be1bb74ad4137a45158f22ce64607"
I20260808 04:48:07.800379 11753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b5fa2c8eb943188c5ab1c2a43c0ff0" candidate_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54e62f5b24844fe8385bd18754d3919"
I20260808 04:48:07.800496 11548 raft_consensus.cc:3037] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:07.800503 11753 raft_consensus.cc:3037] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:07.801856 11548 raft_consensus.cc:2454] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e21ba4f65ec48c0a229ff1cd8fc8b84 in term 1.
I20260808 04:48:07.801888 11753 raft_consensus.cc:2454] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e21ba4f65ec48c0a229ff1cd8fc8b84 in term 1.
I20260808 04:48:07.802166 11254 leader_election.cc:302] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 395be1bb74ad4137a45158f22ce64607, 3e21ba4f65ec48c0a229ff1cd8fc8b84; no voters:
I20260808 04:48:07.802423 11420 raft_consensus.cc:2781] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:48:07.802610 11420 raft_consensus.cc:686] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 LEADER]: Becoming Leader. State: Replica: 3e21ba4f65ec48c0a229ff1cd8fc8b84, State: Running, Role: LEADER
I20260808 04:48:07.802911 11420 consensus_queue.cc:237] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:07.803085 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.798878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:48657 (local address 127.22.2.131:33225)
0808 04:48:07.799000 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.799003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.799013 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:48:07.799045 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.799047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.799080 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.799169 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:07.799559 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.800228 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.801773 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.802336 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.802435 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.802499 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.802898 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.802973 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 04:48:07.803478 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.798476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:33225 (local address 127.22.2.130:48657)
0808 04:48:07.798728 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.798742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:07.798832 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:07.799183 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:07.799186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:07.799202 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:07.799333 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.799344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.800356 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.800359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.801606 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.801619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.802606 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.802610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.802732 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.802770 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 04:48:07.803251 (+ 481us) client_negotiation.cc:240] Negotiation successful
0808 04:48:07.803330 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260808 04:48:07.803895 11753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b5fa2c8eb943188c5ab1c2a43c0ff0" candidate_uuid: "395be1bb74ad4137a45158f22ce64607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54e62f5b24844fe8385bd18754d3919" is_pre_election: true
I20260808 04:48:07.804082 11753 raft_consensus.cc:2379] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 395be1bb74ad4137a45158f22ce64607 in current term 1: Already voted for candidate 3e21ba4f65ec48c0a229ff1cd8fc8b84 in this term.
I20260808 04:48:07.804513 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.799459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45129 (local address 127.22.2.130:55753)
0808 04:48:07.800076 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.800091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:07.800223 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:07.800620 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:07.800623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:07.800645 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:07.800783 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.800793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.801784 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.801789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.802925 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:07.802935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.804037 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.804040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:07.804154 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.804176 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 04:48:07.804285 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 04:48:07.804352 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":70,"threads_started":1}
I20260808 04:48:07.804585 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.799573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:55753 (local address 127.22.2.129:45129)
0808 04:48:07.799881 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.799885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.800159 (+ 274us) server_negotiation.cc:407] Connection header received
0808 04:48:07.800343 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.800346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.800393 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.800454 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:07.800929 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.801617 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.803066 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.803703 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.803805 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.803927 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.804317 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.804411 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":93,"threads_started":1}
I20260808 04:48:07.805104 11344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b5fa2c8eb943188c5ab1c2a43c0ff0" candidate_uuid: "395be1bb74ad4137a45158f22ce64607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" is_pre_election: true
I20260808 04:48:07.805889 11450 leader_election.cc:302] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 395be1bb74ad4137a45158f22ce64607; no voters: 3e21ba4f65ec48c0a229ff1cd8fc8b84, b54e62f5b24844fe8385bd18754d3919
I20260808 04:48:07.806188 11856 raft_consensus.cc:2726] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 04:48:07.806432 11139 catalog_manager.cc:5495] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } health_report { overall_health: HEALTHY } } }
I20260808 04:48:07.880261 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.876388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.22.2.129:45129)
0808 04:48:07.876603 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:07.876610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:07.876628 (+ 18us) server_negotiation.cc:407] Connection header received
0808 04:48:07.876873 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:07.876875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:07.876936 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:07.877055 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:48:07.877339 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.877851 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.878715 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:07.878941 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:07.879065 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:07.879360 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:48:07.879833 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:48:07.879972 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:07.880039 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 04:48:07.880100 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 04:48:07.884858 11753 raft_consensus.cc:1261] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 1 FOLLOWER]: Refusing update from remote peer 3e21ba4f65ec48c0a229ff1cd8fc8b84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:07.884888 11548 raft_consensus.cc:1261] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Refusing update from remote peer 3e21ba4f65ec48c0a229ff1cd8fc8b84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:07.885625 11863 consensus_queue.cc:1035] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:07.885838 11420 consensus_queue.cc:1035] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:07.894838 11868 mvcc.cc:205] Tried to move back new op lower bound from 7316129742369173504 to 7316129742043889664. Current Snapshot: MvccSnapshot[applied={T|T < 7316129742369173504}]
I20260808 04:48:07.903383 11136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:48:07.903533 11136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:48:07.906075 11136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Sending DeleteTablet for 3 replicas of tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0
I20260808 04:48:07.906744 11323 tablet_service.cc:1514] Processing DeleteTablet for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39374
I20260808 04:48:07.906872 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 11196
I20260808 04:48:07.906852 11726 tablet_service.cc:1514] Processing DeleteTablet for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:42498
I20260808 04:48:07.909610 11515 tablet_service.cc:1514] Processing DeleteTablet for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:32864
I20260808 04:48:07.910923 11881 tablet_replica.cc:330] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: stopping tablet replica
I20260808 04:48:07.911307 11881 raft_consensus.cc:2229] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:07.911747 11881 raft_consensus.cc:2258] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:07.913622 11878 tablet_replica.cc:330] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: stopping tablet replica
I20260808 04:48:07.913966 11878 raft_consensus.cc:2229] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:07.914297 11878 raft_consensus.cc:2258] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:07.914557 11881 ts_tablet_manager.cc:1905] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:07.916292 11878 ts_tablet_manager.cc:1905] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:07.918790 11878 ts_tablet_manager.cc:1918] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:07.918913 11878 log.cc:1196] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/e9b5fa2c8eb943188c5ab1c2a43c0ff0
I20260808 04:48:07.919051 11881 ts_tablet_manager.cc:1918] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:07.919186 11881 log.cc:1196] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/e9b5fa2c8eb943188c5ab1c2a43c0ff0
I20260808 04:48:07.919265 11878 ts_tablet_manager.cc:1939] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P b54e62f5b24844fe8385bd18754d3919: Deleting consensus metadata
I20260808 04:48:07.920045 11881 ts_tablet_manager.cc:1939] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 395be1bb74ad4137a45158f22ce64607: Deleting consensus metadata
I20260808 04:48:07.920475 11119 catalog_manager.cc:4841] TS b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225): tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0 (table test-table [id=23f28acda6ff49db8d07869477cc5657]) successfully deleted
I20260808 04:48:07.921373 11118 catalog_manager.cc:4841] TS 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707): tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0 (table test-table [id=23f28acda6ff49db8d07869477cc5657]) successfully deleted
W20260808 04:48:07.927141 11118 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:45129 (error 108)
I20260808 04:48:07.927176 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 11419
I20260808 04:48:07.932010 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:07.931613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:45129 (local address 127.0.0.1:39402)
0808 04:48:07.931851 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:07.931919 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:45129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
W20260808 04:48:07.932230 11118 catalog_manager.cc:4584] TS 3e21ba4f65ec48c0a229ff1cd8fc8b84 (127.22.2.129:45129): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9b5fa2c8eb943188c5ab1c2a43c0ff0: Network error: Client connection negotiation failed: client connection to 127.22.2.129:45129: connect: Connection refused (error 111)
I20260808 04:48:07.936993 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 11619
I20260808 04:48:07.950179 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 11092
I20260808 04:48:07.963663 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42233
--webserver_interface=127.22.2.190
--webserver_port=35517
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:08.077466 11884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:08.077719 11884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:08.077790 11884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:08.081838 11884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:08.081894 11884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:08.081914 11884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:08.081931 11884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:08.081947 11884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:08.087575 11884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=35517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:08.088761 11884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:08.089825 11884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:08.099877 11894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:08.100365 11892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:08.100607 11897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:08.100924 11884 server_base.cc:1034] running on GCE node
I20260808 04:48:08.101593 11884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:08.102761 11884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:08.103963 11884 hybrid_clock.cc:648] HybridClock initialized: now 1786164488103944 us; error 38 us; skew 500 ppm
Aug 08 04:48:08 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164488, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:08.106843 11884 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:48:08.107726 11884 webserver.cc:460] Webserver started at http://127.22.2.190:35517/ using document root <none> and password file <none>
I20260808 04:48:08.108319 11884 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:08.108418 11884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:08.112226 11884 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:48:08.114461 11912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:08.115634 11884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 04:48:08.115779 11884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "e54513a6d81e4608b09252c2b95ec61b"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "c98349ddc74df5d1e8337b059b0161c6"
server_key_iv: "fd8f1b3b9dfb5e92251ed41c862cdfa4"
server_key_version: "encryptionkey@0"
I20260808 04:48:08.116329 11884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:08.145198 11884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:08.148809 11884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:08.149037 11884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:08.156607 11884 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42233
I20260808 04:48:08.156908 11993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42233 every 8 connection(s)
I20260808 04:48:08.157752 11884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:48:08.160928 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 11884
I20260808 04:48:08.161458 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:45129
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:42233
--webserver_port=36241
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:08.162600 11994 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:48:08.164733 11994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Bootstrap starting.
I20260808 04:48:08.183571 11994 log.cc:824] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:08.195597 11994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:48:08.196012 11994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b: Bootstrap complete.
I20260808 04:48:08.200248 11994 raft_consensus.cc:348] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:08.200730 11994 raft_consensus.cc:729] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54513a6d81e4608b09252c2b95ec61b, State: Initialized, Role: FOLLOWER
I20260808 04:48:08.201300 11994 consensus_queue.cc:260] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:08.201455 11994 raft_consensus.cc:388] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:48:08.201547 11994 raft_consensus.cc:482] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:48:08.201663 11994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 2 FOLLOWER]: Advancing to term 3
I20260808 04:48:08.203614 11994 raft_consensus.cc:504] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:08.203979 11994 leader_election.cc:302] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e54513a6d81e4608b09252c2b95ec61b; no voters:
I20260808 04:48:08.204355 11994 leader_election.cc:288] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 04:48:08.204447 12002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 04:48:08.204671 12002 raft_consensus.cc:686] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [term 3 LEADER]: Becoming Leader. State: Replica: e54513a6d81e4608b09252c2b95ec61b, State: Running, Role: LEADER
I20260808 04:48:08.204980 12002 consensus_queue.cc:237] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } }
I20260808 04:48:08.205682 11994 sys_catalog.cc:564] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:08.206542 12003 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e54513a6d81e4608b09252c2b95ec61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } } }
I20260808 04:48:08.206686 12003 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:08.207053 12004 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e54513a6d81e4608b09252c2b95ec61b. Latest consensus state: current_term: 3 leader_uuid: "e54513a6d81e4608b09252c2b95ec61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54513a6d81e4608b09252c2b95ec61b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42233 } } }
I20260808 04:48:08.207154 12004 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54513a6d81e4608b09252c2b95ec61b [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:08.207245 12014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:48:08.212738 12014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:48:08.213680 12014 catalog_manager.cc:1259] Loaded cluster ID: b07eea6058584cf5a0de91d0029c5b94
I20260808 04:48:08.213752 12014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 04:48:08.214892 12014 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f592b8c19ad google::LogMessage::Fail() at ??:0
@ 0x7f592b8c488a google::LogMessage::SendToLog() at ??:0
@ 0x7f592b8c14de google::LogMessage::Flush() at ??:0
@ 0x7f592b8c2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5936200fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5936201d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f59361fe579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5936242640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f592db133d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f592db13c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f592db153f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f592db04512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f592f1486db start_thread at ??:0
@ 0x7f592a3bf61f clone at ??:0
W20260808 04:48:08.299618 11996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:08.299996 11996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:08.300094 11996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:08.305049 11996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:08.305145 11996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:08.305244 11996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:48:08.310758 11996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:45129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=36241
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:08.311892 11996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:08.313016 11996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:08.322829 12053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:08.322814 12055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:08.322830 12051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:08.323347 11996 server_base.cc:1034] running on GCE node
I20260808 04:48:08.323773 11996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:08.324411 11996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:08.325605 11996 hybrid_clock.cc:648] HybridClock initialized: now 1786164488325576 us; error 47 us; skew 500 ppm
Aug 08 04:48:08 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164488, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:08.328517 11996 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:48:08.329523 11996 webserver.cc:460] Webserver started at http://127.22.2.129:36241/ using document root <none> and password file <none>
I20260808 04:48:08.330102 11996 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:08.330207 11996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:08.334249 11996 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:48:08.336581 12067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:08.337906 11996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:48:08.338043 11996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "ccd20a855e0e3100df1b3084988ee93a"
server_key_iv: "b4a86bc8ccfee5e30eced0cf77b390f5"
server_key_version: "encryptionkey@0"
I20260808 04:48:08.338590 11996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:08.353708 11996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:08.357116 11996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:08.357337 11996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:08.357949 11996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:08.359105 12081 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 04:48:08.364765 11996 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 04:48:08.364827 11996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 04:48:08.364902 11996 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 04:48:08.366601 11996 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 04:48:08.366662 11996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 04:48:08.366926 12081 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Bootstrap starting.
I20260808 04:48:08.378063 11996 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:45129
I20260808 04:48:08.378278 12247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:45129 every 8 connection(s)
I20260808 04:48:08.379287 11996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:48:08.379973 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 11996
I20260808 04:48:08.380441 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:39707
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:42233
--webserver_port=37487
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:08.390789 12081 log.cc:824] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:08.400322 12081 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:48:08.400784 12081 tablet_bootstrap.cc:492] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Bootstrap complete.
I20260808 04:48:08.402277 12081 ts_tablet_manager.cc:1397] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Time spent bootstrapping tablet: real 0.036s user 0.025s sys 0.007s
I20260808 04:48:08.405555 12081 raft_consensus.cc:348] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:08.406116 12081 raft_consensus.cc:729] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e21ba4f65ec48c0a229ff1cd8fc8b84, State: Initialized, Role: FOLLOWER
I20260808 04:48:08.406649 12081 consensus_queue.cc:260] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:08.407725 12081 ts_tablet_manager.cc:1428] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 04:48:08.444792 12253 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:48:08.382550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:37967)
0808 04:48:08.383791 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.383854 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:08.384910 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:08.443785 (+ 58875us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":985,"thread_start_us":148,"threads_started":1}
W20260808 04:48:08.445343 12248 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:42233 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:42233: Client connection negotiation failed: client connection to 127.22.2.190:42233: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 04:48:08.446152 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:08.445840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:41735)
0808 04:48:08.445994 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.446078 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 04:48:08.446936 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:08.446681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:39043)
0808 04:48:08.446807 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.446867 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 04:48:08.447134 12248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:48:08.503597 12252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:08.503849 12252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:08.503893 12252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:08.507812 12252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:08.507890 12252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:08.507951 12252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:48:08.512997 12252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:39707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=37487
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:08.513932 12252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:08.514891 12252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:08.524964 12264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:08.524999 12266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:08.524964 12263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:08.525425 12252 server_base.cc:1034] running on GCE node
I20260808 04:48:08.525847 12252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:08.526557 12252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:08.527753 12252 hybrid_clock.cc:648] HybridClock initialized: now 1786164488527722 us; error 40 us; skew 500 ppm
Aug 08 04:48:08 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164488, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:08.535662 12252 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:48:08.536703 12252 webserver.cc:460] Webserver started at http://127.22.2.130:37487/ using document root <none> and password file <none>
I20260808 04:48:08.537292 12252 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:08.537357 12252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:08.541181 12252 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 04:48:08.543437 12274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:08.544775 12252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 04:48:08.544914 12252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "395be1bb74ad4137a45158f22ce64607"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "dcc2a36559708428a9388f201ba5b63f"
server_key_iv: "890c6b09a840e99e537f052df92b9a96"
server_key_version: "encryptionkey@0"
I20260808 04:48:08.545490 12252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:08.567668 12252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:08.571404 12252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:08.571620 12252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:08.572254 12252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:08.573273 12252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:08.573354 12252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:08.573417 12252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:08.573450 12252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:08.583237 12252 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:39707
I20260808 04:48:08.583271 12463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:39707 every 8 connection(s)
I20260808 04:48:08.584422 12252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:48:08.588934 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:08.586881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:50187)
0808 04:48:08.587417 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.587544 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":160,"threads_started":1}
W20260808 04:48:08.589453 12465 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:42233 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:42233: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
I20260808 04:48:08.589609 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 12252
I20260808 04:48:08.590088 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:33225
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:42233
--webserver_port=46841
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:08.590173 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:08.589890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:55247)
0808 04:48:08.590036 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.590099 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 04:48:08.591094 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:08.590808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:36981)
0808 04:48:08.590950 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.591011 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 04:48:08.591305 12465 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:48:08.705456 12469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:08.705700 12469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:08.705741 12469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:08.709882 12469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:08.709941 12469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:08.710033 12469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:48:08.715677 12469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:33225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=46841
--tserver_master_addrs=127.22.2.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:08.716828 12469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:08.717859 12469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:08.727972 12480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:08.727972 12478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:08.728155 12469 server_base.cc:1034] running on GCE node
W20260808 04:48:08.728410 12476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:08.728859 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:08.729516 12469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:08.730681 12469 hybrid_clock.cc:648] HybridClock initialized: now 1786164488730665 us; error 28 us; skew 500 ppm
Aug 08 04:48:08 dist-test-slave-04bb krb5kdc[10274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164488, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:08.733597 12469 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:48:08.734445 12469 webserver.cc:460] Webserver started at http://127.22.2.131:46841/ using document root <none> and password file <none>
I20260808 04:48:08.734983 12469 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:08.735044 12469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:08.738837 12469 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 04:48:08.740908 12490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:08.742215 12469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 04:48:08.742345 12469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "b54e62f5b24844fe8385bd18754d3919"
format_stamp: "Formatted at 2026-08-08 04:48:05 on dist-test-slave-04bb"
server_key: "fdc86abbe8b35d1fe46fd5c1ef1cab9e"
server_key_iv: "c55ec9e2d1e14f4c002622a5cf57a90b"
server_key_version: "encryptionkey@0"
I20260808 04:48:08.742965 12469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:08.760491 12469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:08.763906 12469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:08.764122 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:08.764845 12469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:08.765702 12469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:08.765786 12469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:08.765880 12469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:08.765933 12469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:08.775023 12469 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:33225
I20260808 04:48:08.775046 12656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:33225 every 8 connection(s)
I20260808 04:48:08.776043 12469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:48:08.777525 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 12469
I20260808 04:48:08.780427 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:08.778585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:48671)
0808 04:48:08.779109 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.779255 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":134,"threads_started":1}
W20260808 04:48:08.780990 12657 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:42233 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:42233: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
I20260808 04:48:08.781580 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:08.781371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:58975)
0808 04:48:08.781482 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.781527 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 04:48:08.782192 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:08.781995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:49473)
0808 04:48:08.782098 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:08.782140 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 04:48:08.782389 12657 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 04:48:09.448153 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:09.447700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:60451)
0808 04:48:09.447980 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:09.448037 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":106,"threads_started":1}
I20260808 04:48:09.592308 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:09.591853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:50161)
0808 04:48:09.592112 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:09.592171 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260808 04:48:09.783317 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:09.782913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:49465)
0808 04:48:09.783155 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:09.783209 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":68,"threads_started":1}
I20260808 04:48:09.856168 12668 raft_consensus.cc:482] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:09.856413 12668 raft_consensus.cc:504] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:09.857925 12668 leader_election.cc:288] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
Aug 08 04:48:09 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164488, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.131@KRBTEST.COM
Aug 08 04:48:09 dist-test-slave-04bb krb5kdc[10274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164488, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.130@KRBTEST.COM
I20260808 04:48:09.872452 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:09.858192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:33225 (local address 127.22.2.129:34675)
0808 04:48:09.858425 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:09.858440 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:09.858550 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:09.860297 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:09.860302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:09.860883 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:09.861309 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:09.861323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:09.862689 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:09.862692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:09.863583 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:09.863593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:09.864336 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:09.864340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:09.864598 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:09.865309 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:09.865326 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:09.866901 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:09.869539 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:09.869545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:09.869560 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:09.869927 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:09.870392 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:09.870394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:09.870396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:09.870568 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:09.871360 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:09.871366 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:09.871621 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 04:48:09.871829 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 04:48:09.872190 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":54,"spinlock_wait_cycles":26880}
I20260808 04:48:09.872504 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:09.858200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:39707 (local address 127.22.2.129:42683)
0808 04:48:09.858631 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:09.858648 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:09.858735 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:09.860295 (+ 1560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:09.860301 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:09.860858 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:09.861308 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:09.861323 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:09.862650 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:09.862653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:09.863582 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:09.863593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:09.864338 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:09.864342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:09.864595 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:09.865326 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:09.865338 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:09.867276 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:09.870229 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:09.870233 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:09.870236 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:09.870494 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:09.870886 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:09.870889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:09.870891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:09.870982 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:09.871661 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:09.871664 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:09.871750 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 04:48:09.871903 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 04:48:09.872290 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":340,"spinlock_wait_cycles":1920,"thread_start_us":79,"threads_started":1}
I20260808 04:48:09.872547 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:09.858399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34675 (local address 127.22.2.131:33225)
0808 04:48:09.858841 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:09.858869 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:09.858905 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:48:09.859562 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:09.859567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:09.859911 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:09.860226 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:09.861493 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:09.862488 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:09.863731 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:09.864198 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:09.864381 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:09.867165 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:09.867174 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:09.867193 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:09.867230 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:09.869409 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:09.870067 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:09.870073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:09.870077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:09.870251 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:09.870674 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:09.870677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:09.870678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:09.871228 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:09.871344 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:09.872005 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 04:48:09.872353 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":96,"threads_started":1}
I20260808 04:48:09.872496 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:09.858403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:42683 (local address 127.22.2.130:39707)
0808 04:48:09.858834 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:09.858858 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:09.858897 (+ 39us) server_negotiation.cc:407] Connection header received
0808 04:48:09.859564 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:09.859569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:09.859910 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:09.860235 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:09.861495 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:09.862457 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:09.863718 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:09.864198 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:09.864393 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:09.867467 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:09.867471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:09.867483 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:09.867513 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:09.870115 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:09.870611 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:09.870618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:09.870626 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:09.870780 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:09.871079 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:09.871082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:09.871083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:09.871540 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:09.871676 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:09.872053 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 04:48:09.872327 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":115,"threads_started":1}
W20260808 04:48:09.875428 12076 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
W20260808 04:48:09.875634 12074 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
I20260808 04:48:09.875716 12074 leader_election.cc:302] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e21ba4f65ec48c0a229ff1cd8fc8b84; no voters: 395be1bb74ad4137a45158f22ce64607, b54e62f5b24844fe8385bd18754d3919
I20260808 04:48:09.875977 12668 raft_consensus.cc:2726] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:48:10.449441 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:10.449006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:60391)
0808 04:48:10.449282 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:10.449341 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
I20260808 04:48:10.593497 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:10.593063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:51369)
0808 04:48:10.593343 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:10.593402 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":102,"threads_started":1}
I20260808 04:48:10.784475 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:10.784072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:58633)
0808 04:48:10.784332 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:10.784388 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260808 04:48:11.450735 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:11.450315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:44709)
0808 04:48:11.450577 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:11.450635 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":76,"threads_started":1}
I20260808 04:48:11.594811 12681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:11.594314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:57929)
0808 04:48:11.594597 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:11.594661 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
I20260808 04:48:11.750679 12682 raft_consensus.cc:482] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:11.750840 12682 raft_consensus.cc:504] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:11.751300 12682 leader_election.cc:288] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
W20260808 04:48:11.752220 12074 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
W20260808 04:48:11.752336 12076 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
I20260808 04:48:11.752444 12076 leader_election.cc:302] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e21ba4f65ec48c0a229ff1cd8fc8b84; no voters: 395be1bb74ad4137a45158f22ce64607, b54e62f5b24844fe8385bd18754d3919
I20260808 04:48:11.752627 12682 raft_consensus.cc:2726] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:48:11.785621 12683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:11.785215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:46837)
0808 04:48:11.785463 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:11.785517 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":78,"threads_started":1}
I20260808 04:48:12.451915 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:12.451488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:45593)
0808 04:48:12.451759 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:12.451816 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
I20260808 04:48:12.596014 12686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:12.595606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:51329)
0808 04:48:12.595862 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:12.595918 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260808 04:48:12.786804 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:12.786365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:33427)
0808 04:48:12.786652 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:12.786710 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":93,"threads_started":1}
I20260808 04:48:13.453300 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:13.452851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:55581)
0808 04:48:13.453148 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:13.453203 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":85,"threads_started":1}
I20260808 04:48:13.493510 12690 raft_consensus.cc:482] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:13.493670 12690 raft_consensus.cc:504] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:13.494210 12690 leader_election.cc:288] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
W20260808 04:48:13.495106 12074 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
W20260808 04:48:13.495216 12076 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
I20260808 04:48:13.495328 12076 leader_election.cc:302] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e21ba4f65ec48c0a229ff1cd8fc8b84; no voters: 395be1bb74ad4137a45158f22ce64607, b54e62f5b24844fe8385bd18754d3919
I20260808 04:48:13.495514 12690 raft_consensus.cc:2726] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:48:13.597394 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:13.596958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:59737)
0808 04:48:13.597241 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:13.597299 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":98,"threads_started":1}
I20260808 04:48:13.787997 12697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:13.787528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:51879)
0808 04:48:13.787832 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:13.787886 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":90,"threads_started":1}
I20260808 04:48:14.454499 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:14.454064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:59549)
0808 04:48:14.454348 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:14.454402 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":78,"threads_started":1}
I20260808 04:48:14.598680 12700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:14.598227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:33511)
0808 04:48:14.598516 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:14.598573 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
I20260808 04:48:14.789224 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:14.788759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:48057)
0808 04:48:14.789036 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:14.789094 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260808 04:48:15.456024 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:15.455532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:39469)
0808 04:48:15.455869 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:15.455927 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":120,"threads_started":1}
I20260808 04:48:15.600018 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:15.599555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:60219)
0808 04:48:15.599868 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:15.599924 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":81,"threads_started":1}
I20260808 04:48:15.790421 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:15.789975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:48417)
0808 04:48:15.790277 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:15.790331 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":84,"threads_started":1}
I20260808 04:48:16.457338 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:16.456885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:40389)
0808 04:48:16.457153 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:16.457213 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260808 04:48:16.601476 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:16.601030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:50585)
0808 04:48:16.601317 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:16.601380 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
I20260808 04:48:16.791653 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:16.791221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:37387)
0808 04:48:16.791502 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:16.791560 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260808 04:48:17.458668 12713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:17.458197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:56437)
0808 04:48:17.458494 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:17.458550 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":79,"threads_started":1}
I20260808 04:48:17.602703 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:17.602250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:52701)
0808 04:48:17.602550 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:17.602607 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":103,"threads_started":1}
I20260808 04:48:17.793102 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:17.792641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:43965)
0808 04:48:17.792945 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:17.793006 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":97,"threads_started":1}
I20260808 04:48:18.025002 12720 raft_consensus.cc:482] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:18.025141 12720 raft_consensus.cc:504] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:18.025563 12720 leader_election.cc:288] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
W20260808 04:48:18.026355 12074 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
W20260808 04:48:18.026459 12076 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
I20260808 04:48:18.026528 12076 leader_election.cc:302] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e21ba4f65ec48c0a229ff1cd8fc8b84; no voters: 395be1bb74ad4137a45158f22ce64607, b54e62f5b24844fe8385bd18754d3919
I20260808 04:48:18.026676 12720 raft_consensus.cc:2726] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:48:18.460107 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:18.459593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:46785)
0808 04:48:18.459906 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:18.459983 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":101,"threads_started":1}
I20260808 04:48:18.604043 12722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:18.603585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:48779)
0808 04:48:18.603892 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:18.603948 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":102,"threads_started":1}
I20260808 04:48:18.794382 12724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:18.793972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:32853)
0808 04:48:18.794228 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:18.794285 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":81,"threads_started":1}
I20260808 04:48:19.461416 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:19.460985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:60345)
0808 04:48:19.461254 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:19.461317 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260808 04:48:19.605372 12726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:19.604951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:56883)
0808 04:48:19.605206 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:19.605261 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260808 04:48:19.795689 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:19.795250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:54575)
0808 04:48:19.795535 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:19.795592 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":96,"threads_started":1}
I20260808 04:48:20.462666 12728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:20.462226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:43313)
0808 04:48:20.462513 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:20.462568 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":89,"threads_started":1}
I20260808 04:48:20.606740 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:20.606316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:45641)
0808 04:48:20.606594 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:20.606649 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":101,"threads_started":1}
I20260808 04:48:20.737212 12730 raft_consensus.cc:482] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:20.737353 12730 raft_consensus.cc:504] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395be1bb74ad4137a45158f22ce64607" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39707 } } peers { permanent_uuid: "b54e62f5b24844fe8385bd18754d3919" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33225 } } peers { permanent_uuid: "3e21ba4f65ec48c0a229ff1cd8fc8b84" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 45129 } }
I20260808 04:48:20.737847 12730 leader_election.cc:288] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707), b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225)
W20260808 04:48:20.738691 12076 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b54e62f5b24844fe8385bd18754d3919 (127.22.2.131:33225): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
W20260808 04:48:20.738809 12074 leader_election.cc:341] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395be1bb74ad4137a45158f22ce64607 (127.22.2.130:39707): Not found: Tablet not found: e9b5fa2c8eb943188c5ab1c2a43c0ff0
I20260808 04:48:20.738927 12074 leader_election.cc:302] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e21ba4f65ec48c0a229ff1cd8fc8b84; no voters: 395be1bb74ad4137a45158f22ce64607, b54e62f5b24844fe8385bd18754d3919
I20260808 04:48:20.739127 12730 raft_consensus.cc:2726] T e9b5fa2c8eb943188c5ab1c2a43c0ff0 P 3e21ba4f65ec48c0a229ff1cd8fc8b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:48:20.797050 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:20.796595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:55621)
0808 04:48:20.796892 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:20.796953 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260808 04:48:21.464068 12737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:21.463539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:41129)
0808 04:48:21.463891 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:21.463966 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":98,"threads_started":1}
I20260808 04:48:21.607955 12739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:21.607533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:48243)
0808 04:48:21.607787 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:21.607857 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260808 04:48:21.798342 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:21.797888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:35319)
0808 04:48:21.798170 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:21.798228 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260808 04:48:22.465525 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:22.465074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:44821)
0808 04:48:22.465365 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:22.465428 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":104,"threads_started":1}
I20260808 04:48:22.609258 12743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:22.608853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:53223)
0808 04:48:22.609104 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:22.609160 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
I20260808 04:48:22.799595 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:22.799129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:45725)
0808 04:48:22.799436 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:22.799495 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":105,"threads_started":1}
I20260808 04:48:23.467031 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:23.466606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.129:38193)
0808 04:48:23.466878 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:23.466932 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":74,"threads_started":1}
I20260808 04:48:23.610570 12746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:23.610152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.130:59371)
0808 04:48:23.610427 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:23.610476 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":74,"threads_started":1}
I20260808 04:48:23.781391 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 11996
I20260808 04:48:23.795815 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 12252
I20260808 04:48:23.802029 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:23.801575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42233 (local address 127.22.2.131:42225)
0808 04:48:23.801871 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:23.801933 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:42233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":86,"threads_started":1}
I20260808 04:48:23.807113 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 12469
2026-08-08T04:48:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20151 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:48:23 dist-test-slave-04bb krb5kdc[12751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:48:23 dist-test-slave-04bb krb5kdc[12751](info): set up 2 sockets
Aug 08 04:48:23 dist-test-slave-04bb krb5kdc[12751](info): commencing operation
krb5kdc: starting...
W20260808 04:48:25.055567 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:48:25 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:48:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:48:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:48:25.197901 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41909
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:25.321034 12769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:25.321329 12769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:25.321401 12769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:25.325380 12769 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 04:48:25.325448 12769 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 04:48:25.325474 12769 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 04:48:25.325503 12769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:25.325547 12769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:25.325584 12769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:25.325639 12769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:25.325668 12769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:25.330930 12769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:25.331969 12769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:25.333041 12769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:25.342106 12778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:25.342084 12776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:25.342069 12780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:25.342165 12769 server_base.cc:1034] running on GCE node
I20260808 04:48:25.342849 12769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:25.343977 12769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:25.345146 12769 hybrid_clock.cc:648] HybridClock initialized: now 1786164505345126 us; error 34 us; skew 500 ppm
Aug 08 04:48:25 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164505, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:25.347997 12769 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:48:25.348924 12769 webserver.cc:460] Webserver started at http://127.22.2.190:35885/ using document root <none> and password file <none>
I20260808 04:48:25.349457 12769 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:25.349514 12769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:25.349676 12769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:25.351655 12769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "824a116c015d4063a92c9e5fed6931b0"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "75a16a7cd043527bd0dc3f4a04d306ca"
server_key_iv: "b86238821f2e807949db5bb5d3af93fa"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.352170 12769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "824a116c015d4063a92c9e5fed6931b0"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "75a16a7cd043527bd0dc3f4a04d306ca"
server_key_iv: "b86238821f2e807949db5bb5d3af93fa"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.355866 12769 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:48:25.358855 12788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:25.360395 12769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 04:48:25.360522 12769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "824a116c015d4063a92c9e5fed6931b0"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "75a16a7cd043527bd0dc3f4a04d306ca"
server_key_iv: "b86238821f2e807949db5bb5d3af93fa"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.360662 12769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:25.379038 12769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:25.382419 12769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:25.382627 12769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:25.389750 12769 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41909
I20260808 04:48:25.390036 12870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41909 every 8 connection(s)
I20260808 04:48:25.390761 12769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:48:25.393730 12872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:25.397570 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 12769
I20260808 04:48:25.397668 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:48:25.397962 22538 external_mini_cluster.cc:1428] Setting key 5f8b4056fa697851faf615602ef92ce0
I20260808 04:48:25.399596 12872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Bootstrap starting.
I20260808 04:48:25.402129 12872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:25.402911 12872 log.cc:824] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:25.405222 12872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: No bootstrap required, opened a new log
Aug 08 04:48:25 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:25.409441 12872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:48:25.409734 12872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:25.409802 12872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824a116c015d4063a92c9e5fed6931b0, State: Initialized, Role: FOLLOWER
I20260808 04:48:25.410291 12872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:48:25.410475 12872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:48:25.410557 12872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:48:25.410661 12872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:25.411866 12872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:48:25.412240 12872 leader_election.cc:302] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824a116c015d4063a92c9e5fed6931b0; no voters:
I20260808 04:48:25.412792 12872 leader_election.cc:288] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:48:25.412958 12883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:48:25.413237 12883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 1 LEADER]: Becoming Leader. State: Replica: 824a116c015d4063a92c9e5fed6931b0, State: Running, Role: LEADER
I20260808 04:48:25.413472 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:25.399445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47062 (local address 127.22.2.190:41909)
0808 04:48:25.399885 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:25.399902 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:25.399939 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:48:25.400644 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:25.400663 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:25.401005 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:25.401287 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:25.402296 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.403331 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.404318 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.405062 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.405232 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:25.407798 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:25.407818 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:25.407855 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:25.407898 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:25.409956 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:25.410452 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:25.410458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:25.410464 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:25.410618 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:25.410954 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:25.410957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:25.410959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:25.411479 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:25.411668 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:25.411952 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 04:48:25.412271 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":110,"threads_started":1}
I20260808 04:48:25.413589 12883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:48:25.414124 12872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:25.415477 12884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "824a116c015d4063a92c9e5fed6931b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } } }
I20260808 04:48:25.415616 12884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:25.415851 12885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 824a116c015d4063a92c9e5fed6931b0. Latest consensus state: current_term: 1 leader_uuid: "824a116c015d4063a92c9e5fed6931b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } } }
I20260808 04:48:25.415968 12885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:25.416144 12898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:48:25.419736 12898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:48:25.425414 12898 catalog_manager.cc:1347] Generated new cluster ID: c7dee115e1fa4a4f9679402b93677af5
I20260808 04:48:25.425484 12898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:48:25.435103 12898 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:48:25.436044 12898 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:48:25.444016 12898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Generated new TSK 0
I20260808 04:48:25.444877 12898 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:48:25.483448 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41909
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:25.597767 12914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:25.597992 12914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:25.598032 12914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:25.601977 12914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:25.602027 12914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:25.602082 12914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:48:25.607079 12914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:25.608074 12914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:25.609114 12914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:25.619233 12923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:25.619241 12926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:25.619376 12921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:25.619632 12914 server_base.cc:1034] running on GCE node
I20260808 04:48:25.620078 12914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:25.620755 12914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:25.621927 12914 hybrid_clock.cc:648] HybridClock initialized: now 1786164505621889 us; error 56 us; skew 500 ppm
Aug 08 04:48:25 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164505, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:25.624812 12914 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:48:25.625736 12914 webserver.cc:460] Webserver started at http://127.22.2.129:45831/ using document root <none> and password file <none>
I20260808 04:48:25.626348 12914 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:25.626444 12914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:25.626678 12914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:25.628901 12914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "88d88a10195c493da8422cccbc4765ad"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "1cebe297f977634df42aa0dd795c6210"
server_key_iv: "e932c26d56419e0070865a5711ed4b0a"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.629493 12914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "88d88a10195c493da8422cccbc4765ad"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "1cebe297f977634df42aa0dd795c6210"
server_key_iv: "e932c26d56419e0070865a5711ed4b0a"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.633224 12914 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:48:25.635697 12943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:25.637097 12914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:48:25.637238 12914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "88d88a10195c493da8422cccbc4765ad"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "1cebe297f977634df42aa0dd795c6210"
server_key_iv: "e932c26d56419e0070865a5711ed4b0a"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.637352 12914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:25.681737 12914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:25.685048 12914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:25.685268 12914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:25.685933 12914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:25.686841 12914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:25.686918 12914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:25.686995 12914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:25.687037 12914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:25.696343 12914 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33767
I20260808 04:48:25.696378 13116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33767 every 8 connection(s)
I20260808 04:48:25.697291 12914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:48:25.700356 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 12914
I20260808 04:48:25.700491 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:48:25.700863 22538 external_mini_cluster.cc:1428] Setting key 36c1c8bdd35d4967de008af75376483a
Aug 08 04:48:25 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164505, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:48:25.713440 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:25.699301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:36043 (local address 127.22.2.190:41909)
0808 04:48:25.699489 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:25.699493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:25.700340 (+ 847us) server_negotiation.cc:407] Connection header received
0808 04:48:25.701980 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:25.701983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:25.702027 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:25.702101 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:25.703525 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.704365 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.706280 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.706507 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.706594 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:25.709638 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:25.709641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:25.709643 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:25.709667 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:25.711259 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:25.711801 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:25.711804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:25.711805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:25.711882 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:25.712238 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:25.712241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:25.712243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:25.712475 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:25.712633 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:25.713140 (+ 507us) server_negotiation.cc:299] Negotiation successful
0808 04:48:25.713263 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 04:48:25.714284 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:25.699656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:36043)
0808 04:48:25.700162 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:25.700209 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:25.701754 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:25.702347 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:25.702358 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:25.702834 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:25.703361 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:25.703375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.704562 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.704568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:25.706128 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:25.706153 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.706677 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.706684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:25.706925 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:25.707599 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:25.707631 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:25.709457 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:25.711388 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:25.711394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:25.711416 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:25.711671 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:25.711980 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:25.711983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:25.711985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:25.712129 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:25.712634 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:25.712641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:25.712887 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 04:48:25.713078 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 04:48:25.713412 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 04:48:25.715687 13117 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41909
I20260808 04:48:25.715972 13117 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:25.716678 13117 heartbeater.cc:511] Master 127.22.2.190:41909 requested a full tablet report, sending...
I20260808 04:48:25.718353 12819 ts_manager.cc:194] Registered new tserver with Master: 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767)
I20260808 04:48:25.719489 12819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:36043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:48:25.731946 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41909
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:25.850610 13127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:25.850907 13127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:25.850986 13127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:25.855011 13127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:25.855110 13127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:25.855204 13127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:48:25.860689 13127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:25.861765 13127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:25.862816 13127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:25.872586 13133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:25.872648 13136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:25.872856 13134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:25.873082 13127 server_base.cc:1034] running on GCE node
I20260808 04:48:25.873481 13127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:25.874240 13127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:25.875469 13127 hybrid_clock.cc:648] HybridClock initialized: now 1786164505875457 us; error 34 us; skew 500 ppm
Aug 08 04:48:25 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164505, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:25.878723 13127 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:48:25.879739 13127 webserver.cc:460] Webserver started at http://127.22.2.130:41843/ using document root <none> and password file <none>
I20260808 04:48:25.880360 13127 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:25.880456 13127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:25.880734 13127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:25.882900 13127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "90a339c599df4a77ace7dc0e2aca2d46"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "05a6174bd18e070078f457791665a6a4"
server_key_iv: "a5d605d5d931270410bd89d600274bd1"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.883518 13127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "90a339c599df4a77ace7dc0e2aca2d46"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "05a6174bd18e070078f457791665a6a4"
server_key_iv: "a5d605d5d931270410bd89d600274bd1"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.887755 13127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 04:48:25.890787 13145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:25.892288 13127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:48:25.892465 13127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "90a339c599df4a77ace7dc0e2aca2d46"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "05a6174bd18e070078f457791665a6a4"
server_key_iv: "a5d605d5d931270410bd89d600274bd1"
server_key_version: "encryptionkey@0"
I20260808 04:48:25.892679 13127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:25.918454 13127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:25.921917 13127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:25.922163 13127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:25.922802 13127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:25.923682 13127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:25.923754 13127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:25.923820 13127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:25.923878 13127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:25.933415 13127 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41115
I20260808 04:48:25.933439 13323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41115 every 8 connection(s)
I20260808 04:48:25.934345 13127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:48:25.939230 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 13127
I20260808 04:48:25.939325 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:48:25.939656 22538 external_mini_cluster.cc:1428] Setting key 2f8c3d61fba42d2a52de7d533c4f8c8e
Aug 08 04:48:25 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164505, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:25.949457 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:25.936240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:59775 (local address 127.22.2.190:41909)
0808 04:48:25.936398 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:25.936403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:25.937350 (+ 947us) server_negotiation.cc:407] Connection header received
0808 04:48:25.938636 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:25.938639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:25.938688 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:25.938774 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:25.940393 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.941077 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.942345 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.942574 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.942699 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:25.945320 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:25.945324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:25.945326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:25.945349 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:25.947053 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:25.947581 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:25.947587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:25.947591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:25.947697 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:25.948164 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:25.948170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:25.948174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:25.948431 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:25.948576 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:25.949117 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 04:48:25.949254 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 04:48:25.950202 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:25.936736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:59775)
0808 04:48:25.937191 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:25.937236 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:25.938415 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:25.939018 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:25.939027 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:25.939500 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:25.940083 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:25.940103 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.941201 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.941204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:25.942216 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:25.942226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:25.942742 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:25.942746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:25.942918 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:25.943542 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:25.943558 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:25.945151 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:25.947193 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:25.947198 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:25.947210 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:25.947473 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:25.947894 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:25.947897 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:25.947898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:25.948043 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:25.948603 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:25.948608 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:25.948844 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 04:48:25.949017 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 04:48:25.949278 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:48:25.951418 13325 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41909
I20260808 04:48:25.951671 13325 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:25.952176 13325 heartbeater.cc:511] Master 127.22.2.190:41909 requested a full tablet report, sending...
I20260808 04:48:25.953332 12819 ts_manager.cc:194] Registered new tserver with Master: 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:48:25.954144 12819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:59775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:48:25.970054 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41909
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:26.088915 13331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:26.089218 13331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:26.089298 13331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:26.095451 13331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:26.095523 13331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:26.095621 13331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:48:26.102316 13331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:26.103345 13331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:26.104369 13331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:26.114843 13339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:26.115084 13331 server_base.cc:1034] running on GCE node
W20260808 04:48:26.114849 13341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:26.114867 13338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:26.115684 13331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:26.116824 13331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:26.118026 13331 hybrid_clock.cc:648] HybridClock initialized: now 1786164506117991 us; error 49 us; skew 500 ppm
Aug 08 04:48:26 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164506, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:26.121096 13331 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:48:26.122040 13331 webserver.cc:460] Webserver started at http://127.22.2.131:42399/ using document root <none> and password file <none>
I20260808 04:48:26.122641 13331 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:26.122736 13331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:26.122977 13331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:48:26.125077 13331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "ae30ca4cf9a347db807121c92b1778bc"
format_stamp: "Formatted at 2026-08-08 04:48:26 on dist-test-slave-04bb"
server_key: "c94c9876c58a7c3fc0b513a3c99b8805"
server_key_iv: "7771db14789f728099277398a3d075f8"
server_key_version: "encryptionkey@0"
I20260808 04:48:26.125651 13331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "ae30ca4cf9a347db807121c92b1778bc"
format_stamp: "Formatted at 2026-08-08 04:48:26 on dist-test-slave-04bb"
server_key: "c94c9876c58a7c3fc0b513a3c99b8805"
server_key_iv: "7771db14789f728099277398a3d075f8"
server_key_version: "encryptionkey@0"
I20260808 04:48:26.129722 13331 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 04:48:26.132279 13350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:26.133613 13331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:48:26.133749 13331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "ae30ca4cf9a347db807121c92b1778bc"
format_stamp: "Formatted at 2026-08-08 04:48:26 on dist-test-slave-04bb"
server_key: "c94c9876c58a7c3fc0b513a3c99b8805"
server_key_iv: "7771db14789f728099277398a3d075f8"
server_key_version: "encryptionkey@0"
I20260808 04:48:26.133867 13331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:26.150858 13331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:26.153946 13331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:26.154143 13331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:26.154742 13331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:26.155591 13331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:26.155660 13331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:26.155727 13331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:26.155773 13331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:26.165347 13331 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:35187
I20260808 04:48:26.165370 13538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:35187 every 8 connection(s)
I20260808 04:48:26.166213 13331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:48:26.167037 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 13331
I20260808 04:48:26.167178 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:48:26.167506 22538 external_mini_cluster.cc:1428] Setting key e366b25cefa05615ea9f3989e3b1a22f
Aug 08 04:48:26 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164506, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:26.180589 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.168384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:39979 (local address 127.22.2.190:41909)
0808 04:48:26.168663 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:26.168668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:26.169674 (+ 1006us) server_negotiation.cc:407] Connection header received
0808 04:48:26.171213 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:26.171216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:26.171262 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:26.171336 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:26.172575 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.173115 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.174257 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.174480 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.174556 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.177093 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:26.177096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:26.177098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:26.177123 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:26.178608 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:26.179104 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:26.179106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:26.179108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:26.179181 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:26.179522 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:26.179525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:26.179526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:26.179689 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:26.179767 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:26.180318 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 04:48:26.180414 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260808 04:48:26.181803 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.168885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:39979)
0808 04:48:26.169494 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:26.169557 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:26.171007 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:26.171510 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:26.171516 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:26.171877 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:26.172417 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.172431 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.173249 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.173252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.174144 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.174155 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.174637 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.174642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.174799 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.175337 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:26.175355 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:26.176927 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:26.178720 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:26.178725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:26.178741 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:26.179012 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:26.179288 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:26.179290 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:26.179292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:26.179432 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:26.179815 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:26.179822 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:26.180054 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 04:48:26.180241 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 04:48:26.180658 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":192,"threads_started":1}
I20260808 04:48:26.183074 13540 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41909
I20260808 04:48:26.183295 13540 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:26.183748 13540 heartbeater.cc:511] Master 127.22.2.190:41909 requested a full tablet report, sending...
I20260808 04:48:26.184765 12819 ts_manager.cc:194] Registered new tserver with Master: ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187)
I20260808 04:48:26.185360 12819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:39979
I20260808 04:48:26.193783 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:48:26.203361 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.195861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.22.2.190:41909)
0808 04:48:26.196009 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:26.196015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:26.196163 (+ 148us) server_negotiation.cc:407] Connection header received
0808 04:48:26.196342 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:26.196346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:26.196396 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:26.196462 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:26.197193 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.197744 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.198787 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.198998 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.199082 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.200474 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:26.200477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:26.200479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:26.200501 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:26.201818 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:26.202252 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:26.202254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:26.202255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:26.202345 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:26.202582 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:26.202584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:26.202585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:26.202778 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:26.202881 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:26.203087 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 04:48:26.203178 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 04:48:26.206126 12819 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:48:26.209164 12819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:48:26.224380 13555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.218216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33767 (local address 127.0.0.1:52682)
0808 04:48:26.218795 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:26.218849 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:26.219003 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:26.219610 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:26.219616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:26.219639 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:26.219809 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.219819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.221314 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.221318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.222702 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.222712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.223745 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.223749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.223868 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.223910 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 04:48:26.224033 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 04:48:26.224191 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260808 04:48:26.224375 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.218742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52682 (local address 127.22.2.129:33767)
0808 04:48:26.219039 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:26.219047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:26.219078 (+ 31us) server_negotiation.cc:407] Connection header received
0808 04:48:26.219169 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:26.219175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:26.219430 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:26.219986 (+ 556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:26.220145 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.221166 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.222866 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.223603 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.223714 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.223865 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:26.224087 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 04:48:26.224208 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":107,"threads_started":1}
I20260808 04:48:26.224864 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.218216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35187 (local address 127.0.0.1:52536)
0808 04:48:26.219618 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:26.219633 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:26.219826 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:26.220427 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:26.220430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:26.220446 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:26.220586 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.220600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.221882 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.221885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.223156 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.223167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.224421 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.224425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.224525 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.224573 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 04:48:26.224655 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 04:48:26.224724 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"mutex_wait_us":51,"thread_start_us":122,"threads_started":1}
I20260808 04:48:26.224920 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.218313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52536 (local address 127.22.2.131:35187)
0808 04:48:26.219455 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:26.219471 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:26.219735 (+ 264us) server_negotiation.cc:407] Connection header received
0808 04:48:26.219999 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:26.220004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:26.220162 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:26.220263 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:26.220719 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.221739 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.223298 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.224115 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.224218 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.224348 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:26.224690 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 04:48:26.224783 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1019,"thread_start_us":151,"threads_started":1}
I20260808 04:48:26.225801 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.219212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41115 (local address 127.0.0.1:51886)
0808 04:48:26.219923 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:26.219937 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:26.220011 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:26.221230 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:26.221234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:26.221255 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:26.221347 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.221354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.222954 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.222957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.224073 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.224083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.225412 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.225416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.225511 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.225536 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 04:48:26.225591 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 04:48:26.225672 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":102,"threads_started":1}
I20260808 04:48:26.226491 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.219422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.22.2.130:41115)
0808 04:48:26.220786 (+ 1364us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:26.220797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:26.220820 (+ 23us) server_negotiation.cc:407] Connection header received
0808 04:48:26.220893 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:26.220898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:26.221062 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:26.221219 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:26.221456 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.222816 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.224232 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.225234 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.226016 (+ 782us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.226166 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:26.226221 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 04:48:26.226314 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1246,"thread_start_us":131,"threads_started":1}
I20260808 04:48:26.228253 13016 tablet_service.cc:1467] Processing CreateTablet for tablet e200fb98f1d44f46beb3c22ca45e7a3b (DEFAULT_TABLE table=test-table [id=1ba153ebae6041fcadaf91c1305fa7d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:26.228276 13431 tablet_service.cc:1467] Processing CreateTablet for tablet e200fb98f1d44f46beb3c22ca45e7a3b (DEFAULT_TABLE table=test-table [id=1ba153ebae6041fcadaf91c1305fa7d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:26.229462 13016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e200fb98f1d44f46beb3c22ca45e7a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:26.229044 13217 tablet_service.cc:1467] Processing CreateTablet for tablet e200fb98f1d44f46beb3c22ca45e7a3b (DEFAULT_TABLE table=test-table [id=1ba153ebae6041fcadaf91c1305fa7d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:26.229602 13431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e200fb98f1d44f46beb3c22ca45e7a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:26.230062 13217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e200fb98f1d44f46beb3c22ca45e7a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:26.236227 13564 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Bootstrap starting.
I20260808 04:48:26.237807 13566 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Bootstrap starting.
I20260808 04:48:26.239234 13564 tablet_bootstrap.cc:654] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:26.240082 13566 tablet_bootstrap.cc:654] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:26.240201 13564 log.cc:824] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:26.240896 13565 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Bootstrap starting.
I20260808 04:48:26.241567 13566 log.cc:824] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:26.242719 13564 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: No bootstrap required, opened a new log
I20260808 04:48:26.242954 13564 ts_tablet_manager.cc:1397] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 04:48:26.243436 13565 tablet_bootstrap.cc:654] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:26.243856 13566 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: No bootstrap required, opened a new log
I20260808 04:48:26.244062 13566 ts_tablet_manager.cc:1397] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:48:26.244398 13565 log.cc:824] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:26.246587 13565 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: No bootstrap required, opened a new log
I20260808 04:48:26.246784 13565 ts_tablet_manager.cc:1397] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:48:26.247449 13564 raft_consensus.cc:348] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.247715 13564 raft_consensus.cc:374] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:26.247777 13564 raft_consensus.cc:729] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d88a10195c493da8422cccbc4765ad, State: Initialized, Role: FOLLOWER
I20260808 04:48:26.247748 13566 raft_consensus.cc:348] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.248024 13566 raft_consensus.cc:374] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:26.248080 13566 raft_consensus.cc:729] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae30ca4cf9a347db807121c92b1778bc, State: Initialized, Role: FOLLOWER
I20260808 04:48:26.248315 13564 consensus_queue.cc:260] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.248590 13566 consensus_queue.cc:260] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.249116 13117 heartbeater.cc:503] Master 127.22.2.190:41909 was elected leader, sending a full tablet report...
I20260808 04:48:26.249418 13564 ts_tablet_manager.cc:1428] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 04:48:26.249545 13566 ts_tablet_manager.cc:1428] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 04:48:26.249915 13540 heartbeater.cc:503] Master 127.22.2.190:41909 was elected leader, sending a full tablet report...
I20260808 04:48:26.250536 13565 raft_consensus.cc:348] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.250790 13565 raft_consensus.cc:374] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:26.250845 13565 raft_consensus.cc:729] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a339c599df4a77ace7dc0e2aca2d46, State: Initialized, Role: FOLLOWER
I20260808 04:48:26.251259 13565 consensus_queue.cc:260] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.251916 13325 heartbeater.cc:503] Master 127.22.2.190:41909 was elected leader, sending a full tablet report...
I20260808 04:48:26.252223 13565 ts_tablet_manager.cc:1428] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 04:48:26.417141 13541 tablet.cc:2369] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:48:26.435601 13327 tablet.cc:2369] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:48:26.448751 13119 tablet.cc:2369] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:48:26.502244 13573 raft_consensus.cc:482] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:26.502460 13573 raft_consensus.cc:504] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.503780 13573 leader_election.cc:288] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187), 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767)
I20260808 04:48:26.509261 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.504260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:52183 (local address 127.22.2.129:33767)
0808 04:48:26.504403 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:26.504408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:26.504590 (+ 182us) server_negotiation.cc:407] Connection header received
0808 04:48:26.504703 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:26.504706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:26.504784 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:26.504863 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:26.505208 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.506011 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.507861 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.508492 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.508650 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.508707 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:26.509051 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 04:48:26.509126 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 04:48:26.509284 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.504159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:50039 (local address 127.22.2.131:35187)
0808 04:48:26.504332 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:26.504338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:26.504466 (+ 128us) server_negotiation.cc:407] Connection header received
0808 04:48:26.504636 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:26.504639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:26.504707 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:26.504799 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:26.505208 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.506007 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.507861 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.508472 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.508650 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.508707 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:26.509051 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 04:48:26.509126 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 04:48:26.509333 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.503991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35187 (local address 127.22.2.130:50039)
0808 04:48:26.504343 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:26.504360 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:26.504494 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:26.504827 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:26.504849 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:26.504875 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:26.505063 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.505077 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.506131 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.506135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.507717 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.507726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.508650 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.508654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.508842 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.508936 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 04:48:26.509019 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 04:48:26.509139 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":315,"spinlock_wait_cycles":16256,"thread_start_us":98,"threads_started":1}
I20260808 04:48:26.509336 13577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.504124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33767 (local address 127.22.2.130:52183)
0808 04:48:26.504462 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:26.504474 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:26.504601 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:26.504885 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:26.504887 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:26.504926 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:26.505091 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.505098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.506131 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.506135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.507714 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:26.507726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.508650 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.508654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:26.508817 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.508935 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 04:48:26.509019 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 04:48:26.509139 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":239,"spinlock_wait_cycles":1536,"thread_start_us":110,"threads_started":1}
I20260808 04:48:26.509985 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e200fb98f1d44f46beb3c22ca45e7a3b" candidate_uuid: "90a339c599df4a77ace7dc0e2aca2d46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae30ca4cf9a347db807121c92b1778bc" is_pre_election: true
I20260808 04:48:26.510186 13043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e200fb98f1d44f46beb3c22ca45e7a3b" candidate_uuid: "90a339c599df4a77ace7dc0e2aca2d46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d88a10195c493da8422cccbc4765ad" is_pre_election: true
I20260808 04:48:26.510295 13465 raft_consensus.cc:2454] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a339c599df4a77ace7dc0e2aca2d46 in term 0.
I20260808 04:48:26.510509 13043 raft_consensus.cc:2454] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a339c599df4a77ace7dc0e2aca2d46 in term 0.
I20260808 04:48:26.510816 13150 leader_election.cc:302] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a339c599df4a77ace7dc0e2aca2d46, ae30ca4cf9a347db807121c92b1778bc; no voters:
I20260808 04:48:26.511121 13573 raft_consensus.cc:2781] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:48:26.511246 13573 raft_consensus.cc:482] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:48:26.511317 13573 raft_consensus.cc:3037] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:26.512422 13573 raft_consensus.cc:504] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.512830 13573 leader_election.cc:288] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [CANDIDATE]: Term 1 election: Requested vote from peers ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187), 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767)
I20260808 04:48:26.513187 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e200fb98f1d44f46beb3c22ca45e7a3b" candidate_uuid: "90a339c599df4a77ace7dc0e2aca2d46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae30ca4cf9a347db807121c92b1778bc"
I20260808 04:48:26.513201 13043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e200fb98f1d44f46beb3c22ca45e7a3b" candidate_uuid: "90a339c599df4a77ace7dc0e2aca2d46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d88a10195c493da8422cccbc4765ad"
I20260808 04:48:26.513330 13465 raft_consensus.cc:3037] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:26.513348 13043 raft_consensus.cc:3037] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:26.514379 13465 raft_consensus.cc:2454] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a339c599df4a77ace7dc0e2aca2d46 in term 1.
I20260808 04:48:26.514372 13043 raft_consensus.cc:2454] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a339c599df4a77ace7dc0e2aca2d46 in term 1.
I20260808 04:48:26.514736 13150 leader_election.cc:302] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a339c599df4a77ace7dc0e2aca2d46, ae30ca4cf9a347db807121c92b1778bc; no voters:
I20260808 04:48:26.514961 13573 raft_consensus.cc:2781] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:48:26.515218 13573 raft_consensus.cc:686] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 LEADER]: Becoming Leader. State: Replica: 90a339c599df4a77ace7dc0e2aca2d46, State: Running, Role: LEADER
I20260808 04:48:26.515581 13573 consensus_queue.cc:237] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:26.518941 12817 catalog_manager.cc:5495] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "90a339c599df4a77ace7dc0e2aca2d46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } health_report { overall_health: HEALTHY } } }
I20260808 04:48:26.591936 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.588591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.22.2.130:41115)
0808 04:48:26.588729 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:26.588732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:26.588814 (+ 82us) server_negotiation.cc:407] Connection header received
0808 04:48:26.588974 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:26.588977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:26.589038 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:26.589129 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:48:26.589356 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.589851 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.590650 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:26.590862 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:26.590939 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:26.591203 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:48:26.591615 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:48:26.591706 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:26.591781 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 04:48:26.591830 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 04:48:26.598559 13465 raft_consensus.cc:1261] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 1 FOLLOWER]: Refusing update from remote peer 90a339c599df4a77ace7dc0e2aca2d46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:26.598592 13043 raft_consensus.cc:1261] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Refusing update from remote peer 90a339c599df4a77ace7dc0e2aca2d46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:26.599290 13573 consensus_queue.cc:1035] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:26.599493 13578 consensus_queue.cc:1035] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:26.607175 13581 mvcc.cc:205] Tried to move back new op lower bound from 7316129819020455936 to 7316129818690846720. Current Snapshot: MvccSnapshot[applied={T|T < 7316129819020455936}]
I20260808 04:48:26.609112 13583 mvcc.cc:205] Tried to move back new op lower bound from 7316129819020455936 to 7316129818690846720. Current Snapshot: MvccSnapshot[applied={T|T < 7316129819020455936}]
I20260808 04:48:26.609468 13582 mvcc.cc:205] Tried to move back new op lower bound from 7316129819020455936 to 7316129818690846720. Current Snapshot: MvccSnapshot[applied={T|T < 7316129819020455936}]
I20260808 04:48:26.616402 12814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:48:26.616604 12814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:48:26.619897 12814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Sending DeleteTablet for 3 replicas of tablet e200fb98f1d44f46beb3c22ca45e7a3b
I20260808 04:48:26.620672 13431 tablet_service.cc:1514] Processing DeleteTablet for tablet e200fb98f1d44f46beb3c22ca45e7a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:26 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:52536
I20260808 04:48:26.620672 13217 tablet_service.cc:1514] Processing DeleteTablet for tablet e200fb98f1d44f46beb3c22ca45e7a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:26 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:51886
I20260808 04:48:26.621184 13592 tablet_replica.cc:330] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: stopping tablet replica
I20260808 04:48:26.621193 13016 tablet_service.cc:1514] Processing DeleteTablet for tablet e200fb98f1d44f46beb3c22ca45e7a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:26 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:52682
I20260808 04:48:26.621490 13597 tablet_replica.cc:330] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: stopping tablet replica
I20260808 04:48:26.621553 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 12914
I20260808 04:48:26.621673 13599 tablet_replica.cc:330] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: stopping tablet replica
I20260808 04:48:26.622007 13599 raft_consensus.cc:2229] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:26.625556 13597 raft_consensus.cc:2229] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:48:26.622008 13592 raft_consensus.cc:2229] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:26.625998 13592 raft_consensus.cc:2258] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:26.626415 13597 raft_consensus.cc:2258] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:26.629119 13592 ts_tablet_manager.cc:1905] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:26.631963 13592 ts_tablet_manager.cc:1918] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:26.632088 13592 log.cc:1196] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/e200fb98f1d44f46beb3c22ca45e7a3b
I20260808 04:48:26.632431 13592 ts_tablet_manager.cc:1939] T e200fb98f1d44f46beb3c22ca45e7a3b P ae30ca4cf9a347db807121c92b1778bc: Deleting consensus metadata
I20260808 04:48:26.634123 12797 catalog_manager.cc:4841] TS ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187): tablet e200fb98f1d44f46beb3c22ca45e7a3b (table test-table [id=1ba153ebae6041fcadaf91c1305fa7d3]) successfully deleted
I20260808 04:48:26.639127 13597 ts_tablet_manager.cc:1905] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:26.642733 13597 ts_tablet_manager.cc:1918] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:26.642858 13597 log.cc:1196] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/e200fb98f1d44f46beb3c22ca45e7a3b
I20260808 04:48:26.643229 13597 ts_tablet_manager.cc:1939] T e200fb98f1d44f46beb3c22ca45e7a3b P 90a339c599df4a77ace7dc0e2aca2d46: Deleting consensus metadata
I20260808 04:48:26.644708 12791 catalog_manager.cc:4841] TS 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115): tablet e200fb98f1d44f46beb3c22ca45e7a3b (table test-table [id=1ba153ebae6041fcadaf91c1305fa7d3]) successfully deleted
W20260808 04:48:26.646274 12795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:33767 (error 108)
I20260808 04:48:26.646520 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 13127
I20260808 04:48:26.651712 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:26.651342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33767 (local address 127.0.0.1:52692)
0808 04:48:26.651525 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:26.651608 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 04:48:26.652320 12795 catalog_manager.cc:4584] TS 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e200fb98f1d44f46beb3c22ca45e7a3b: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33767: connect: Connection refused (error 111)
I20260808 04:48:26.658094 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 13331
I20260808 04:48:26.669377 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 12769
I20260808 04:48:46.683179 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41909
--webserver_interface=127.22.2.190
--webserver_port=35885
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:48:46.797716 13601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:46.798028 13601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:46.798107 13601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:46.802064 13601 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 04:48:46.802141 13601 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 04:48:46.802171 13601 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 04:48:46.802197 13601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:48:46.802251 13601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:46.802313 13601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:46.802353 13601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:48:46.802389 13601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:48:46.807579 13601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=35885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:46.808712 13601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:46.809825 13601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:46.824558 13611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:46.824606 13614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:46.824608 13601 server_base.cc:1034] running on GCE node
W20260808 04:48:46.824628 13610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:46.825507 13601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:46.826682 13601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:46.827862 13601 hybrid_clock.cc:648] HybridClock initialized: now 1786164526827844 us; error 28 us; skew 500 ppm
Aug 08 04:48:46 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164526, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:46.830876 13601 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:48:46.831801 13601 webserver.cc:460] Webserver started at http://127.22.2.190:35885/ using document root <none> and password file <none>
I20260808 04:48:46.832401 13601 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:46.832471 13601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:46.836501 13601 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260808 04:48:46.838836 13625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:46.840214 13601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:48:46.840389 13601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "824a116c015d4063a92c9e5fed6931b0"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "75a16a7cd043527bd0dc3f4a04d306ca"
server_key_iv: "b86238821f2e807949db5bb5d3af93fa"
server_key_version: "encryptionkey@0"
I20260808 04:48:46.841068 13601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:46.863103 13601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:46.866509 13601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:46.866734 13601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:46.874225 13601 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41909
I20260808 04:48:46.874577 13704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41909 every 8 connection(s)
I20260808 04:48:46.875418 13601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:48:46.880905 13705 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:48:46.881273 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 13601
I20260808 04:48:46.881769 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:33767
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:41909
--webserver_port=45831
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:46.883060 13705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Bootstrap starting.
I20260808 04:48:46.902134 13705 log.cc:824] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:46.911851 13705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:48:46.912264 13705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Bootstrap complete.
I20260808 04:48:46.916625 13705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:48:46.917083 13705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824a116c015d4063a92c9e5fed6931b0, State: Initialized, Role: FOLLOWER
I20260808 04:48:46.917670 13705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:48:46.917830 13705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:48:46.917933 13705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:48:46.918061 13705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:48:46.920063 13705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:48:46.920428 13705 leader_election.cc:302] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824a116c015d4063a92c9e5fed6931b0; no voters:
I20260808 04:48:46.920837 13705 leader_election.cc:288] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 04:48:46.920938 13711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 04:48:46.921173 13711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 2 LEADER]: Becoming Leader. State: Replica: 824a116c015d4063a92c9e5fed6931b0, State: Running, Role: LEADER
I20260808 04:48:46.921554 13711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:48:46.922163 13705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:48:46.923084 13712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "824a116c015d4063a92c9e5fed6931b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } } }
I20260808 04:48:46.923221 13712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:46.923565 13713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 824a116c015d4063a92c9e5fed6931b0. Latest consensus state: current_term: 2 leader_uuid: "824a116c015d4063a92c9e5fed6931b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } } }
I20260808 04:48:46.923663 13713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: This master's current role is: LEADER
I20260808 04:48:46.927784 13730 catalog_manager.cc:1259] Loaded cluster ID: c7dee115e1fa4a4f9679402b93677af5
I20260808 04:48:46.927868 13730 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: loading cluster ID for follower catalog manager: success
I20260808 04:48:46.930401 13730 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: acquiring CA information for follower catalog manager: success
I20260808 04:48:46.932024 13730 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 04:48:46.932200 13724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:48:46.934832 13724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:48:46.935355 13724 catalog_manager.cc:1259] Loaded cluster ID: c7dee115e1fa4a4f9679402b93677af5
I20260808 04:48:46.935424 13724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:48:46.936185 13724 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:48:46.936818 13724 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Loaded TSK: 0
I20260808 04:48:46.954205 13724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Generated new TSK 1
I20260808 04:48:46.954474 13724 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 04:48:47.004428 13708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:47.004778 13708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:47.004868 13708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:47.008881 13708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:47.008957 13708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:47.009038 13708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:48:47.014179 13708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:33767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=45831
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:47.015198 13708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:47.016235 13708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:47.027254 13743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:47.027338 13741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:47.027287 13747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:47.027624 13708 server_base.cc:1034] running on GCE node
I20260808 04:48:47.028060 13708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:47.028808 13708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:47.030025 13708 hybrid_clock.cc:648] HybridClock initialized: now 1786164527029998 us; error 49 us; skew 500 ppm
Aug 08 04:48:47 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164527, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:47.033206 13708 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:48:47.034094 13708 webserver.cc:460] Webserver started at http://127.22.2.129:45831/ using document root <none> and password file <none>
I20260808 04:48:47.034701 13708 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:47.034842 13708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:47.038959 13708 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:48:47.041337 13754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:47.042850 13708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:48:47.043017 13708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "88d88a10195c493da8422cccbc4765ad"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "1cebe297f977634df42aa0dd795c6210"
server_key_iv: "e932c26d56419e0070865a5711ed4b0a"
server_key_version: "encryptionkey@0"
I20260808 04:48:47.043597 13708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:47.060808 13708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:47.064097 13708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:47.064320 13708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:47.065029 13708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:47.066278 13766 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 04:48:47.070724 13708 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 04:48:47.070791 13708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 04:48:47.070840 13708 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 04:48:47.072712 13708 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 04:48:47.072775 13708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260808 04:48:47.073079 13766 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Bootstrap starting.
I20260808 04:48:47.084980 13708 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33767
I20260808 04:48:47.085078 13936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33767 every 8 connection(s)
I20260808 04:48:47.086437 13708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:48:47.090164 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 13708
I20260808 04:48:47.090628 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:41115
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:41909
--webserver_port=41843
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:48:47.096716 13766 log.cc:824] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:48:47 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164527, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:47.105846 13766 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:48:47.106280 13766 tablet_bootstrap.cc:492] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Bootstrap complete.
I20260808 04:48:47.107843 13766 ts_tablet_manager.cc:1397] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Time spent bootstrapping tablet: real 0.035s user 0.023s sys 0.008s
I20260808 04:48:47.108151 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.088697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:50599)
0808 04:48:47.089711 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.089770 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:47.091101 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:47.092632 (+ 1531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:47.092647 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:47.093191 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:47.094122 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.094150 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.095721 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.095725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.096688 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.096700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.097609 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.097614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.097828 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.098667 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:47.098692 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:47.101035 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:47.103965 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:47.103976 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:47.103995 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:47.104448 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:47.104961 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:47.104968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:47.104972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:47.105206 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:47.106071 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:47.106079 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:47.106400 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 04:48:47.106606 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 04:48:47.106953 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":133,"threads_started":1}
I20260808 04:48:47.108439 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.088719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:50599 (local address 127.22.2.190:41909)
0808 04:48:47.089350 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.089380 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.089942 (+ 562us) server_negotiation.cc:407] Connection header received
0808 04:48:47.091629 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.091647 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.092049 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.092431 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:47.094346 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.095539 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.096841 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.097447 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.097670 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.101305 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:47.101312 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:47.101330 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:47.101366 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:47.103811 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.104648 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.104655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.104661 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.104828 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.105356 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.105360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.105362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.105839 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:47.105998 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.106789 (+ 791us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.107146 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":142,"threads_started":1}
I20260808 04:48:47.111477 13937 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41909
I20260808 04:48:47.111728 13937 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:47.111274 13766 raft_consensus.cc:348] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:47.111852 13766 raft_consensus.cc:729] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d88a10195c493da8422cccbc4765ad, State: Initialized, Role: FOLLOWER
I20260808 04:48:47.112319 13937 heartbeater.cc:511] Master 127.22.2.190:41909 requested a full tablet report, sending...
I20260808 04:48:47.112360 13766 consensus_queue.cc:260] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } } peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } }
I20260808 04:48:47.113452 13766 ts_tablet_manager.cc:1428] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 04:48:47.114967 13650 ts_manager.cc:194] Registered new tserver with Master: 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767)
I20260808 04:48:47.118598 13650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:50599
I20260808 04:48:47.121284 13937 heartbeater.cc:503] Master 127.22.2.190:41909 was elected leader, sending a full tablet report...
Aug 08 04:48:47 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164526, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 04:48:47.129769 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.117929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33767 (local address 127.0.0.1:43024)
0808 04:48:47.118912 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.118957 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:47.119098 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:47.119677 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:47.119683 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:47.120091 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:47.120219 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.120226 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.121879 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.121882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.122845 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.122859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.123491 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.123495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.123616 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.124186 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:47.124200 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:47.125770 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:47.127941 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:47.127950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:47.127954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:47.128206 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:47.128573 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:47.128581 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:47.128587 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:47.128696 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:47.129217 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:47.129222 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:47.129331 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 04:48:47.129456 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 04:48:47.129621 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":97,"threads_started":1}
I20260808 04:48:47.130113 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.118035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43024 (local address 127.22.2.129:33767)
0808 04:48:47.118659 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.118670 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.119068 (+ 398us) server_negotiation.cc:407] Connection header received
0808 04:48:47.119260 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.119266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.119484 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.119667 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:47.120630 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.121752 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.122954 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.123364 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.123451 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.125911 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:47.125921 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:47.125934 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:47.125962 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:47.127804 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.128322 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.128328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.128330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.128424 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.128796 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.128799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.128801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.129084 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:47.129217 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.129559 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.129737 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":131,"threads_started":1}
I20260808 04:48:47.131106 13834 tablet_service.cc:1514] Processing DeleteTablet for tablet e200fb98f1d44f46beb3c22ca45e7a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:26 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:43024
I20260808 04:48:47.132257 13948 tablet_replica.cc:330] stopping tablet replica
I20260808 04:48:47.132637 13948 raft_consensus.cc:2229] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:47.132915 13948 raft_consensus.cc:2258] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:47.133903 13948 ts_tablet_manager.cc:1905] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:47.137625 13948 ts_tablet_manager.cc:1918] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:47.137740 13948 log.cc:1196] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal/wals/e200fb98f1d44f46beb3c22ca45e7a3b
I20260808 04:48:47.138041 13948 ts_tablet_manager.cc:1939] T e200fb98f1d44f46beb3c22ca45e7a3b P 88d88a10195c493da8422cccbc4765ad: Deleting consensus metadata
I20260808 04:48:47.139726 13631 catalog_manager.cc:4841] TS 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767): tablet e200fb98f1d44f46beb3c22ca45e7a3b (table test-table [id=1ba153ebae6041fcadaf91c1305fa7d3]) successfully deleted
W20260808 04:48:47.215848 13942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:47.216097 13942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:47.216140 13942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:47.219956 13942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:47.220002 13942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:47.220057 13942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:48:47.225560 13942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:41115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=41843
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:47.226502 13942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:47.227465 13942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:48:47.237412 13942 server_base.cc:1034] running on GCE node
W20260808 04:48:47.237372 13962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:47.237404 13958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:47.237368 13959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:47.237977 13942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:47.238603 13942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:47.239773 13942 hybrid_clock.cc:648] HybridClock initialized: now 1786164527239756 us; error 28 us; skew 500 ppm
Aug 08 04:48:47 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164527, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:47.242550 13942 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:48:47.243378 13942 webserver.cc:460] Webserver started at http://127.22.2.130:41843/ using document root <none> and password file <none>
I20260808 04:48:47.243920 13942 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:47.243992 13942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:47.248250 13942 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:48:47.250450 13974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:47.251783 13942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 04:48:47.251924 13942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "90a339c599df4a77ace7dc0e2aca2d46"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "05a6174bd18e070078f457791665a6a4"
server_key_iv: "a5d605d5d931270410bd89d600274bd1"
server_key_version: "encryptionkey@0"
I20260808 04:48:47.252436 13942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:47.274134 13942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:47.277422 13942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:47.277632 13942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:47.278228 13942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:47.279047 13942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:47.279093 13942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:47.279137 13942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:47.279158 13942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:47.288322 13942 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41115
I20260808 04:48:47.288362 14155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41115 every 8 connection(s)
I20260808 04:48:47.289333 13942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:48:47.290215 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 13942
I20260808 04:48:47.290668 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:35187
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:41909
--webserver_port=42399
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:48:47 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164527, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:47.306072 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.291235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:48641 (local address 127.22.2.190:41909)
0808 04:48:47.291390 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.291393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.292567 (+ 1174us) server_negotiation.cc:407] Connection header received
0808 04:48:47.294808 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.294814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.294882 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.294980 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:47.296270 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.297232 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.298412 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.298863 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.298997 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.301844 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:47.301853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:47.301858 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:47.301900 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:47.303824 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.304372 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.304380 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.304385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.304491 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.304901 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.304905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.304907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.305145 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:47.305261 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.305733 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.305878 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 04:48:47.306888 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.291683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:48641)
0808 04:48:47.292194 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.292239 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:47.294513 (+ 2274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:47.295132 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:47.295139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:47.295502 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:47.296062 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.296077 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.297375 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.297378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.298274 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.298285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.299044 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.299047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.299216 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.299982 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:47.300001 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:47.301625 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:47.303967 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:47.303972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:47.303984 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:47.304252 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:47.304644 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:47.304646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:47.304648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:47.304800 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:47.305264 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:47.305270 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:47.305495 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 04:48:47.305660 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 04:48:47.305924 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":149,"threads_started":1}
I20260808 04:48:47.308172 14156 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41909
I20260808 04:48:47.308441 14156 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:47.308979 14156 heartbeater.cc:511] Master 127.22.2.190:41909 requested a full tablet report, sending...
I20260808 04:48:47.310158 13650 ts_manager.cc:194] Registered new tserver with Master: 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115)
I20260808 04:48:47.310989 13650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:48641
W20260808 04:48:47.410905 14160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:48:47.411302 14160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:48:47.411367 14160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:48:47.415401 14160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:48:47.415460 14160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:48:47.415516 14160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:48:47.420900 14160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:35187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=42399
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:48:47.421859 14160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:48:47.422953 14160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:48:47.433650 14169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:47.433630 14175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:48:47.433689 14171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:48:47.434504 14160 server_base.cc:1034] running on GCE node
I20260808 04:48:47.434978 14160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:48:47.435642 14160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:48:47.436842 14160 hybrid_clock.cc:648] HybridClock initialized: now 1786164527436823 us; error 37 us; skew 500 ppm
Aug 08 04:48:47 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164527, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:48:47.439491 14160 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:48:47.440327 14160 webserver.cc:460] Webserver started at http://127.22.2.131:42399/ using document root <none> and password file <none>
I20260808 04:48:47.440955 14160 fs_manager.cc:359] Metadata directory not provided
I20260808 04:48:47.441072 14160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:48:47.445928 14160 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:48:47.448153 14184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:47.449465 14160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:48:47.449602 14160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "ae30ca4cf9a347db807121c92b1778bc"
format_stamp: "Formatted at 2026-08-08 04:48:26 on dist-test-slave-04bb"
server_key: "c94c9876c58a7c3fc0b513a3c99b8805"
server_key_iv: "7771db14789f728099277398a3d075f8"
server_key_version: "encryptionkey@0"
I20260808 04:48:47.450114 14160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:48:47.472940 14160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:48:47.476075 14160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:48:47.476290 14160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:48:47.476975 14160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:48:47.477844 14160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:48:47.477891 14160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:47.477962 14160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:48:47.477998 14160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:48:47.487844 14160 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:35187
I20260808 04:48:47.487874 14361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:35187 every 8 connection(s)
I20260808 04:48:47.488885 14160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:48:47.489387 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 14160
I20260808 04:48:47.498817 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.490194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.22.2.190:41909)
0808 04:48:47.490337 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.490343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.490554 (+ 211us) server_negotiation.cc:407] Connection header received
0808 04:48:47.490733 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.490737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.490803 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.490889 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:47.491855 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.492690 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.493611 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.493861 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.493977 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.495552 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:47.495556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:47.495558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:47.495584 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:47.497130 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.497565 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.497570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.497572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.497667 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.497959 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.497963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.497965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.498147 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:47.498364 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.498514 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.498627 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 04:48:47 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164527, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:48:47.502892 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.490811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:59259 (local address 127.22.2.190:41909)
0808 04:48:47.491066 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.491070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.492092 (+ 1022us) server_negotiation.cc:407] Connection header received
0808 04:48:47.493564 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.493567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.493617 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.493730 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:47.495015 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.495631 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.496829 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.497055 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.497159 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.499689 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:47.499693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:47.499695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:47.499723 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:47.501172 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.501637 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.501639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.501640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.501716 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.502000 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.502003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.502004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.502153 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:47.502228 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.502672 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.502767 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
I20260808 04:48:47.503683 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.491364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:59259)
0808 04:48:47.491927 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.491983 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:47.493370 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:47.493914 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:47.493921 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:47.494400 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:48:47.494878 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.494890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.495755 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.495759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.496701 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.496711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.497211 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.497216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.497390 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.497949 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:48:47.497965 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:48:47.499492 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:48:47.501286 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:47.501290 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:47.501301 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:47.501544 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:47.501795 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:48:47.501798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:48:47.501799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:48:47.501922 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:48:47.502237 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:48:47.502242 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:48:47.502456 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 04:48:47.502603 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 04:48:47.502845 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":148,"threads_started":1}
I20260808 04:48:47.504838 14362 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41909
I20260808 04:48:47.505066 14362 heartbeater.cc:464] Registering TS with master...
I20260808 04:48:47.505540 14362 heartbeater.cc:511] Master 127.22.2.190:41909 requested a full tablet report, sending...
I20260808 04:48:47.506589 13650 ts_manager.cc:194] Registered new tserver with Master: ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187)
I20260808 04:48:47.507191 13650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:59259
I20260808 04:48:47.512414 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:48:47.516180 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.513039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.22.2.190:41909)
0808 04:48:47.513177 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.513182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.513410 (+ 228us) server_negotiation.cc:407] Connection header received
0808 04:48:47.513528 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.513530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.513626 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.513707 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:48:47.513906 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.514401 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.515181 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.515404 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.515485 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.515750 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:48:47.515863 (+ 113us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 04:48:47.516059 (+ 196us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37384: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 04:48:47.516252 14369 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37384: token expired
W20260808 04:48:47.516647 13543 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.2.190:41909)
I20260808 04:48:47.524250 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.516999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37390 (local address 127.22.2.190:41909)
0808 04:48:47.517117 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.517120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.517131 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:48:47.517209 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.517211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.517242 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.517302 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:48:47.517819 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.518278 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.518998 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.519221 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.519313 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.520773 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:48:47.520776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:48:47.520778 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:48:47.520800 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:48:47.522250 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.522796 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.522800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.522802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.522893 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:48:47.523242 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:48:47.523246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:48:47.523247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:48:47.523497 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:48:47.523710 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.523914 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.524051 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 04:48:47.526795 13650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:48:47.530555 13650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:48:47.538795 13834 tablet_service.cc:1467] Processing CreateTablet for tablet 079a7773e81041e8938c9df3141ba819 (DEFAULT_TABLE table=test-table [id=2848bd7d1d524319a9a70315bf423bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:47.539489 13834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079a7773e81041e8938c9df3141ba819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:47.542537 13766 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Bootstrap starting.
I20260808 04:48:47.543498 13766 tablet_bootstrap.cc:654] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:47.543807 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.537701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41115 (local address 127.0.0.1:54492)
0808 04:48:47.537895 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.537910 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:47.538015 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:47.539323 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:47.539326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:47.539345 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:47.539543 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.539552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.541445 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.541451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.542254 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.542264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.543470 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.543473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.543544 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.543582 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 04:48:47.543644 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 04:48:47.543705 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":75}
I20260808 04:48:47.544026 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.538217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.22.2.130:41115)
0808 04:48:47.538767 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.538785 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.538823 (+ 38us) server_negotiation.cc:407] Connection header received
0808 04:48:47.538914 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.538920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.539148 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.539331 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:47.539842 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.541328 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.542383 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.543352 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.543480 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.543648 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.543725 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.543837 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":160,"threads_started":1}
I20260808 04:48:47.545328 13766 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: No bootstrap required, opened a new log
I20260808 04:48:47.545454 13766 ts_tablet_manager.cc:1397] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 04:48:47.545878 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.537997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35187 (local address 127.0.0.1:56694)
0808 04:48:47.538284 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.538299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:47.538406 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:47.539353 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:47.539357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:47.539378 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:47.539544 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.539552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.541294 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.541298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.542518 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.542529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.545418 (+ 2889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.545422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.545536 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.545573 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 04:48:47.545658 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 04:48:47.545743 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":4,"thread_start_us":133,"threads_started":1}
I20260808 04:48:47.545994 13766 raft_consensus.cc:348] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.546123 13766 raft_consensus.cc:374] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:47.546159 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.538458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.22.2.131:35187)
0808 04:48:47.538828 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.538841 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.538861 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:48:47.538921 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.538925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.539149 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.539705 (+ 556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:47.539890 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.541160 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.544391 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.545271 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.545393 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.545532 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.545791 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.545893 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":162,"threads_started":1}
I20260808 04:48:47.546161 13766 raft_consensus.cc:729] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d88a10195c493da8422cccbc4765ad, State: Initialized, Role: FOLLOWER
I20260808 04:48:47.546304 13766 consensus_queue.cc:260] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.546568 13766 ts_tablet_manager.cc:1428] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:48:47.547660 14044 tablet_service.cc:1467] Processing CreateTablet for tablet 079a7773e81041e8938c9df3141ba819 (DEFAULT_TABLE table=test-table [id=2848bd7d1d524319a9a70315bf423bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:47.548822 14044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079a7773e81041e8938c9df3141ba819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:47.549168 14248 tablet_service.cc:1467] Processing CreateTablet for tablet 079a7773e81041e8938c9df3141ba819 (DEFAULT_TABLE table=test-table [id=2848bd7d1d524319a9a70315bf423bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:48:47.550024 14248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079a7773e81041e8938c9df3141ba819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:48:47.556391 14385 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: Bootstrap starting.
I20260808 04:48:47.557448 14387 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: Bootstrap starting.
I20260808 04:48:47.559213 14387 tablet_bootstrap.cc:654] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:47.559597 14385 tablet_bootstrap.cc:654] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: Neither blocks nor log segments found. Creating new log.
I20260808 04:48:47.560359 14387 log.cc:824] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:47.560359 14385 log.cc:824] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: Log is configured to *not* fsync() on all Append() calls
I20260808 04:48:47.562431 14387 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: No bootstrap required, opened a new log
I20260808 04:48:47.562646 14387 ts_tablet_manager.cc:1397] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:48:47.562811 14385 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: No bootstrap required, opened a new log
I20260808 04:48:47.563038 14385 ts_tablet_manager.cc:1397] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260808 04:48:47.565773 14387 raft_consensus.cc:348] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.566037 14387 raft_consensus.cc:374] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:47.566119 14387 raft_consensus.cc:729] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae30ca4cf9a347db807121c92b1778bc, State: Initialized, Role: FOLLOWER
I20260808 04:48:47.566219 14385 raft_consensus.cc:348] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.566507 14385 raft_consensus.cc:374] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:48:47.566576 14385 raft_consensus.cc:729] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a339c599df4a77ace7dc0e2aca2d46, State: Initialized, Role: FOLLOWER
I20260808 04:48:47.566668 14387 consensus_queue.cc:260] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.567082 14385 consensus_queue.cc:260] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.567445 14362 heartbeater.cc:503] Master 127.22.2.190:41909 was elected leader, sending a full tablet report...
I20260808 04:48:47.567757 14387 ts_tablet_manager.cc:1428] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 04:48:47.567755 14156 heartbeater.cc:503] Master 127.22.2.190:41909 was elected leader, sending a full tablet report...
I20260808 04:48:47.568152 14385 ts_tablet_manager.cc:1428] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260808 04:48:47.568634 14391 raft_consensus.cc:482] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:48:47.568826 14391 raft_consensus.cc:504] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.570202 14391 leader_election.cc:288] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115), 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767)
I20260808 04:48:47.574612 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.570965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:44457 (local address 127.22.2.130:41115)
0808 04:48:47.571063 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.571069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.571084 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:48:47.571127 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.571135 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.571179 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.571277 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:47.571603 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.572357 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.573519 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.574095 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.574231 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.574285 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.574421 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.574500 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 04:48:47.574630 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.570363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41115 (local address 127.22.2.131:44457)
0808 04:48:47.570699 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.570726 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:47.570811 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:47.571304 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:47.571310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:47.571335 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:47.571474 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.571482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.572455 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.572460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.573394 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.573403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.574217 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.574220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.574306 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.574328 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 04:48:47.574394 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 04:48:47.574486 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":1}
I20260808 04:48:47.575122 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.570665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:59913 (local address 127.22.2.129:33767)
0808 04:48:47.570791 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.570796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.571020 (+ 224us) server_negotiation.cc:407] Connection header received
0808 04:48:47.571155 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.571159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.571236 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.571359 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:48:47.571766 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.572573 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.573725 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.574383 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.574508 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.574664 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.574906 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.575009 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:48:47.575256 14082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079a7773e81041e8938c9df3141ba819" candidate_uuid: "ae30ca4cf9a347db807121c92b1778bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a339c599df4a77ace7dc0e2aca2d46" is_pre_election: true
I20260808 04:48:47.575541 14082 raft_consensus.cc:2454] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae30ca4cf9a347db807121c92b1778bc in term 0.
I20260808 04:48:47.575594 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.570539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33767 (local address 127.22.2.131:59913)
0808 04:48:47.570899 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.570914 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:48:47.571053 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:48:47.571472 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:48:47.571476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:48:47.571492 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:48:47.571632 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.571644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.572720 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.572730 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.573595 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:48:47.573605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.574514 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.574518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:48:47.574622 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.574659 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 04:48:47.575390 (+ 731us) client_negotiation.cc:240] Negotiation successful
0808 04:48:47.575480 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":23,"thread_start_us":107,"threads_started":1}
I20260808 04:48:47.576035 14185 leader_election.cc:302] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a339c599df4a77ace7dc0e2aca2d46, ae30ca4cf9a347db807121c92b1778bc; no voters:
I20260808 04:48:47.576263 13867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079a7773e81041e8938c9df3141ba819" candidate_uuid: "ae30ca4cf9a347db807121c92b1778bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d88a10195c493da8422cccbc4765ad" is_pre_election: true
I20260808 04:48:47.576325 14391 raft_consensus.cc:2781] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:48:47.576434 14391 raft_consensus.cc:482] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:48:47.576496 14391 raft_consensus.cc:3037] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:47.576612 13867 raft_consensus.cc:2454] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae30ca4cf9a347db807121c92b1778bc in term 0.
I20260808 04:48:47.577585 14391 raft_consensus.cc:504] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.577912 14391 leader_election.cc:288] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [CANDIDATE]: Term 1 election: Requested vote from peers 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115), 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767)
I20260808 04:48:47.578286 13867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079a7773e81041e8938c9df3141ba819" candidate_uuid: "ae30ca4cf9a347db807121c92b1778bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d88a10195c493da8422cccbc4765ad"
I20260808 04:48:47.578279 14082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079a7773e81041e8938c9df3141ba819" candidate_uuid: "ae30ca4cf9a347db807121c92b1778bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a339c599df4a77ace7dc0e2aca2d46"
I20260808 04:48:47.578398 13867 raft_consensus.cc:3037] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:47.578398 14082 raft_consensus.cc:3037] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:48:47.579651 14082 raft_consensus.cc:2454] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae30ca4cf9a347db807121c92b1778bc in term 1.
I20260808 04:48:47.579651 13867 raft_consensus.cc:2454] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae30ca4cf9a347db807121c92b1778bc in term 1.
I20260808 04:48:47.579957 14188 leader_election.cc:302] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88d88a10195c493da8422cccbc4765ad, ae30ca4cf9a347db807121c92b1778bc; no voters:
I20260808 04:48:47.580160 14391 raft_consensus.cc:2781] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:48:47.580400 14391 raft_consensus.cc:686] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 1 LEADER]: Becoming Leader. State: Replica: ae30ca4cf9a347db807121c92b1778bc, State: Running, Role: LEADER
I20260808 04:48:47.580756 14391 consensus_queue.cc:237] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:48:47.583789 13649 catalog_manager.cc:5495] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc reported cstate change: term changed from 0 to 1, leader changed from <none> to ae30ca4cf9a347db807121c92b1778bc (127.22.2.131). New cstate: current_term: 1 leader_uuid: "ae30ca4cf9a347db807121c92b1778bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } health_report { overall_health: UNKNOWN } } }
W20260808 04:48:47.587677 13938 tablet.cc:2369] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:48:47.599289 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.595363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.22.2.131:35187)
0808 04:48:47.595492 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 04:48:47.595499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:48:47.595521 (+ 22us) server_negotiation.cc:407] Connection header received
0808 04:48:47.595740 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:48:47.595743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:48:47.595808 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:48:47.595930 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:48:47.596202 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.596839 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.597757 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:48:47.598004 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:48:47.598124 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:48:47.598459 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:48:47.598855 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:48:47.599002 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 04:48:47.599072 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 04:48:47.599129 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:48:47.605029 13867 raft_consensus.cc:1261] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Refusing update from remote peer ae30ca4cf9a347db807121c92b1778bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:47.605034 14082 raft_consensus.cc:1261] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 FOLLOWER]: Refusing update from remote peer ae30ca4cf9a347db807121c92b1778bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:48:47.605712 14391 consensus_queue.cc:1035] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:47.605918 14395 consensus_queue.cc:1035] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:48:47.611670 14400 mvcc.cc:205] Tried to move back new op lower bound from 7316129905063354368 to 7316129904973709312. Current Snapshot: MvccSnapshot[applied={T|T < 7316129905063354368}]
I20260808 04:48:47.614861 14402 mvcc.cc:205] Tried to move back new op lower bound from 7316129905063354368 to 7316129904973709312. Current Snapshot: MvccSnapshot[applied={T|T < 7316129905063354368}]
I20260808 04:48:47.621086 13646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:48:47.621264 13646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:48:47.623919 13646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Sending DeleteTablet for 3 replicas of tablet 079a7773e81041e8938c9df3141ba819
I20260808 04:48:47.624667 14044 tablet_service.cc:1514] Processing DeleteTablet for tablet 079a7773e81041e8938c9df3141ba819 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:47 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:54492
I20260808 04:48:47.624699 14248 tablet_service.cc:1514] Processing DeleteTablet for tablet 079a7773e81041e8938c9df3141ba819 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:47 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:56694
I20260808 04:48:47.624884 13834 tablet_service.cc:1514] Processing DeleteTablet for tablet 079a7773e81041e8938c9df3141ba819 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:48:47 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:43024
I20260808 04:48:47.625069 13948 tablet_replica.cc:330] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: stopping tablet replica
I20260808 04:48:47.625207 13948 raft_consensus.cc:2229] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:47.625334 13948 raft_consensus.cc:2258] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:47.625381 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 13708
I20260808 04:48:47.630820 14419 tablet_replica.cc:330] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: stopping tablet replica
I20260808 04:48:47.631145 14419 raft_consensus.cc:2229] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:48:47.631464 14419 raft_consensus.cc:2258] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:47.632265 14418 tablet_replica.cc:330] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: stopping tablet replica
I20260808 04:48:47.632629 14418 raft_consensus.cc:2229] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:48:47.633033 14418 raft_consensus.cc:2258] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:48:47.633069 14419 ts_tablet_manager.cc:1905] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:47.636023 14418 ts_tablet_manager.cc:1905] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:48:47.636307 14419 ts_tablet_manager.cc:1918] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:47.636472 14419 log.cc:1196] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/079a7773e81041e8938c9df3141ba819
I20260808 04:48:47.636963 14419 ts_tablet_manager.cc:1939] T 079a7773e81041e8938c9df3141ba819 P 90a339c599df4a77ace7dc0e2aca2d46: Deleting consensus metadata
I20260808 04:48:47.638379 13629 catalog_manager.cc:4841] TS 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115): tablet 079a7773e81041e8938c9df3141ba819 (table test-table [id=2848bd7d1d524319a9a70315bf423bf5]) successfully deleted
I20260808 04:48:47.639074 14418 ts_tablet_manager.cc:1918] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:48:47.639201 14418 log.cc:1196] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/079a7773e81041e8938c9df3141ba819
I20260808 04:48:47.639504 14418 ts_tablet_manager.cc:1939] T 079a7773e81041e8938c9df3141ba819 P ae30ca4cf9a347db807121c92b1778bc: Deleting consensus metadata
I20260808 04:48:47.640516 13632 catalog_manager.cc:4841] TS ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187): tablet 079a7773e81041e8938c9df3141ba819 (table test-table [id=2848bd7d1d524319a9a70315bf423bf5]) successfully deleted
W20260808 04:48:47.643069 13631 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:33767 (error 108)
I20260808 04:48:47.644116 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:48:47.643819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33767 (local address 127.0.0.1:43032)
0808 04:48:47.643960 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 04:48:47.644033 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 04:48:47.644286 13631 catalog_manager.cc:4584] TS 88d88a10195c493da8422cccbc4765ad (127.22.2.129:33767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 079a7773e81041e8938c9df3141ba819: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33767: connect: Connection refused (error 111)
I20260808 04:48:47.644814 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 13942
I20260808 04:48:47.664304 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 14160
I20260808 04:48:47.679327 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 13601
I20260808 04:49:07.690541 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41909
--webserver_interface=127.22.2.190
--webserver_port=35885
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:07.805696 14421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:07.805991 14421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:07.806035 14421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:07.810441 14421 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 04:49:07.810494 14421 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 04:49:07.810510 14421 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 04:49:07.810525 14421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:07.810544 14421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:07.810561 14421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:07.810577 14421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:07.810599 14421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:07.815891 14421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=35885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:07.816999 14421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:07.817982 14421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:07.826848 14439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:07.826838 14434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:07.826838 14430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:07.827096 14421 server_base.cc:1034] running on GCE node
I20260808 04:49:07.827786 14421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:07.829087 14421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:07.830282 14421 hybrid_clock.cc:648] HybridClock initialized: now 1786164547830256 us; error 36 us; skew 500 ppm
Aug 08 04:49:07 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164547, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:07.839783 14421 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:49:07.840910 14421 webserver.cc:460] Webserver started at http://127.22.2.190:35885/ using document root <none> and password file <none>
I20260808 04:49:07.841516 14421 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:07.841614 14421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:07.845660 14421 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:49:07.848151 14451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:07.849670 14421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:49:07.849848 14421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "824a116c015d4063a92c9e5fed6931b0"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "75a16a7cd043527bd0dc3f4a04d306ca"
server_key_iv: "b86238821f2e807949db5bb5d3af93fa"
server_key_version: "encryptionkey@0"
I20260808 04:49:07.850440 14421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:07.875391 14421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:07.879166 14421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:07.879448 14421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:07.887681 14421 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41909
I20260808 04:49:07.888039 14537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41909 every 8 connection(s)
I20260808 04:49:07.888944 14421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:07.893651 14538 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:49:07.895593 14538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Bootstrap starting.
I20260808 04:49:07.898818 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 14421
I20260808 04:49:07.899282 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:33767
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:41909
--webserver_port=45831
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:49:07.913592 14538 log.cc:824] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:07.925115 14538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:49:07.925520 14538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Bootstrap complete.
I20260808 04:49:07.929723 14538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:49:07.930121 14538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824a116c015d4063a92c9e5fed6931b0, State: Initialized, Role: FOLLOWER
I20260808 04:49:07.930658 14538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:49:07.930811 14538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:07.930923 14538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:07.931074 14538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 2 FOLLOWER]: Advancing to term 3
I20260808 04:49:07.932992 14538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:49:07.933327 14538 leader_election.cc:302] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824a116c015d4063a92c9e5fed6931b0; no voters:
I20260808 04:49:07.933658 14538 leader_election.cc:288] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 04:49:07.933753 14545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 04:49:07.933967 14545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [term 3 LEADER]: Becoming Leader. State: Replica: 824a116c015d4063a92c9e5fed6931b0, State: Running, Role: LEADER
I20260808 04:49:07.934324 14545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } }
I20260808 04:49:07.934923 14538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:07.935719 14546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "824a116c015d4063a92c9e5fed6931b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } } }
I20260808 04:49:07.935864 14546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:07.936619 14547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 824a116c015d4063a92c9e5fed6931b0. Latest consensus state: current_term: 3 leader_uuid: "824a116c015d4063a92c9e5fed6931b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824a116c015d4063a92c9e5fed6931b0" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41909 } } }
I20260808 04:49:07.936733 14547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:07.937244 14558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:07.941660 14558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:07.942220 14558 catalog_manager.cc:1259] Loaded cluster ID: c7dee115e1fa4a4f9679402b93677af5
I20260808 04:49:07.942296 14558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:07.944097 14558 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:07.944829 14558 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Loaded TSK: 0
I20260808 04:49:07.944914 14558 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 824a116c015d4063a92c9e5fed6931b0: Loaded TSK: 1
F20260808 04:49:07.945603 14558 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc7ce7b59ad google::LogMessage::Fail() at ??:0
@ 0x7fc7ce7b888a google::LogMessage::SendToLog() at ??:0
@ 0x7fc7ce7b54de google::LogMessage::Flush() at ??:0
@ 0x7fc7ce7b6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc7d147918d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc7d1471f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc7d911a7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc7d90f499c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc7d90f5178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fc7d9136df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fc7d915582b std::function<>::operator()() at ??:0
@ 0x7fc7d90f4b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fc7d90f5f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc7d90f2579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fc7d9136640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc7d0a073d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc7d0a07c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fc7d0a093f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc7d09f8512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc7d203c6db start_thread at ??:0
@ 0x7fc7cd2b361f clone at ??:0
W20260808 04:49:08.066409 14543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:08.066790 14543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:08.066864 14543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:08.073029 14543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:08.073118 14543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:08.073213 14543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:49:08.081235 14543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:33767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=45831
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:08.082640 14543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:08.084064 14543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:08.098394 14618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:08.098662 14616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:08.098399 14620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:08.098951 14543 server_base.cc:1034] running on GCE node
I20260808 04:49:08.099472 14543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:08.100466 14543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:08.101819 14543 hybrid_clock.cc:648] HybridClock initialized: now 1786164548101784 us; error 53 us; skew 500 ppm
Aug 08 04:49:08 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164548, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:08.105546 14543 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:49:08.106437 14543 webserver.cc:460] Webserver started at http://127.22.2.129:45831/ using document root <none> and password file <none>
I20260808 04:49:08.107067 14543 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:08.107133 14543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:08.111284 14543 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:49:08.113641 14628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:08.115011 14543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:49:08.115161 14543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "88d88a10195c493da8422cccbc4765ad"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "1cebe297f977634df42aa0dd795c6210"
server_key_iv: "e932c26d56419e0070865a5711ed4b0a"
server_key_version: "encryptionkey@0"
I20260808 04:49:08.115710 14543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:08.142539 14543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:08.146255 14543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:08.146479 14543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:08.147135 14543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:08.148415 14636 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 04:49:08.154342 14543 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 04:49:08.154448 14543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 04:49:08.154572 14543 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 04:49:08.156453 14543 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 04:49:08.156534 14543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 04:49:08.156888 14636 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Bootstrap starting.
I20260808 04:49:08.168627 14543 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33767
I20260808 04:49:08.169003 14794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33767 every 8 connection(s)
I20260808 04:49:08.169853 14543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:49:08.178556 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 14543
I20260808 04:49:08.179021 14636 log.cc:824] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:08.179080 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:41115
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:41909
--webserver_port=41843
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:49:08.188616 14636 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:49:08.189073 14636 tablet_bootstrap.cc:492] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Bootstrap complete.
I20260808 04:49:08.190555 14636 ts_tablet_manager.cc:1397] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.007s
I20260808 04:49:08.193888 14636 raft_consensus.cc:348] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:49:08.194561 14636 raft_consensus.cc:729] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d88a10195c493da8422cccbc4765ad, State: Initialized, Role: FOLLOWER
I20260808 04:49:08.195117 14636 consensus_queue.cc:260] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:49:08.196309 14636 ts_tablet_manager.cc:1428] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 04:49:08.213701 14799 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:49:08.172074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:54883)
0808 04:49:08.172661 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.172723 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:08.173642 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:08.212847 (+ 39205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":173,"threads_started":1}
W20260808 04:49:08.214264 14795 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:41909 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:41909: Client connection negotiation failed: client connection to 127.22.2.190:41909: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 04:49:08.215094 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:08.214838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:53971)
0808 04:49:08.214975 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.215039 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 04:49:08.215858 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:08.215670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:54841)
0808 04:49:08.215772 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.215813 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 04:49:08.216104 14795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:49:08.306494 14804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:08.306731 14804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:08.306772 14804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:08.310731 14804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:08.310782 14804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:08.310838 14804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:49:08.315909 14804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:41115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=41843
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:08.316946 14804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:08.317947 14804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:08.327977 14815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:08.328078 14816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:08.328073 14818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:08.328308 14804 server_base.cc:1034] running on GCE node
I20260808 04:49:08.328811 14804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:08.329437 14804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:08.330618 14804 hybrid_clock.cc:648] HybridClock initialized: now 1786164548330588 us; error 43 us; skew 500 ppm
Aug 08 04:49:08 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164548, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:08.333478 14804 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:49:08.334316 14804 webserver.cc:460] Webserver started at http://127.22.2.130:41843/ using document root <none> and password file <none>
I20260808 04:49:08.334836 14804 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:08.334895 14804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:08.338589 14804 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:49:08.340615 14826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:08.341801 14804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:08.341923 14804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "90a339c599df4a77ace7dc0e2aca2d46"
format_stamp: "Formatted at 2026-08-08 04:48:25 on dist-test-slave-04bb"
server_key: "05a6174bd18e070078f457791665a6a4"
server_key_iv: "a5d605d5d931270410bd89d600274bd1"
server_key_version: "encryptionkey@0"
I20260808 04:49:08.342693 14804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:08.366943 14804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:08.370473 14804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:08.370694 14804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:08.371328 14804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:08.372238 14804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:08.372289 14804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:08.372330 14804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:08.372380 14804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:08.382069 14804 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41115
I20260808 04:49:08.382098 15000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41115 every 8 connection(s)
I20260808 04:49:08.383083 14804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:49:08.387071 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:08.385289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:33767)
0808 04:49:08.385790 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.385913 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":135,"threads_started":1}
I20260808 04:49:08.387596 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 14804
W20260808 04:49:08.387719 15001 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:41909 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:41909: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
I20260808 04:49:08.388118 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:35187
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:41909
--webserver_port=42399
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:49:08.388437 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:08.388141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:53075)
0808 04:49:08.388311 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.388366 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 04:49:08.389088 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:08.388876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:35653)
0808 04:49:08.388984 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.389031 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 04:49:08.389279 15001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:49:08.501786 15004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:08.502027 15004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:08.502067 15004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:08.506023 15004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:08.506072 15004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:08.506127 15004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:49:08.511204 15004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:35187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=42399
--tserver_master_addrs=127.22.2.190:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:08.512156 15004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:08.513196 15004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:08.523339 15015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:08.523452 15011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:08.523340 15012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:08.523947 15004 server_base.cc:1034] running on GCE node
I20260808 04:49:08.524643 15004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:08.525511 15004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:08.526711 15004 hybrid_clock.cc:648] HybridClock initialized: now 1786164548526686 us; error 41 us; skew 500 ppm
Aug 08 04:49:08 dist-test-slave-04bb krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164548, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:08.529592 15004 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:49:08.530490 15004 webserver.cc:460] Webserver started at http://127.22.2.131:42399/ using document root <none> and password file <none>
I20260808 04:49:08.531064 15004 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:08.531129 15004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:08.535169 15004 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 04:49:08.537387 15028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:08.538765 15004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:49:08.538905 15004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "ae30ca4cf9a347db807121c92b1778bc"
format_stamp: "Formatted at 2026-08-08 04:48:26 on dist-test-slave-04bb"
server_key: "c94c9876c58a7c3fc0b513a3c99b8805"
server_key_iv: "7771db14789f728099277398a3d075f8"
server_key_version: "encryptionkey@0"
I20260808 04:49:08.539450 15004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:08.551892 15004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:08.554996 15004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:08.555207 15004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:08.555797 15004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:08.556648 15004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:08.556695 15004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:08.556759 15004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:08.556793 15004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:08.566349 15004 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:35187
I20260808 04:49:08.566381 15210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:35187 every 8 connection(s)
I20260808 04:49:08.567469 15004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:49:08.571725 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:08.569897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:55221)
0808 04:49:08.570407 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.570524 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":132,"threads_started":1}
W20260808 04:49:08.572238 15211 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:41909 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:41909: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
I20260808 04:49:08.572835 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:08.572621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:55119)
0808 04:49:08.572728 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.572780 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 04:49:08.573434 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:08.573253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:43745)
0808 04:49:08.573348 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:08.573389 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 04:49:08.573642 15211 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 04:49:08.575791 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 15004
I20260808 04:49:09.217172 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:09.216753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:48093)
0808 04:49:09.217000 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:09.217054 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260808 04:49:09.390228 15220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:09.389841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:48981)
0808 04:49:09.390076 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:09.390132 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":68,"threads_started":1}
I20260808 04:49:09.574591 15222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:09.574152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:46121)
0808 04:49:09.574429 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:09.574486 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
I20260808 04:49:09.957643 15224 raft_consensus.cc:482] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:09.957858 15224 raft_consensus.cc:504] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:49:09.958971 15224 leader_election.cc:288] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115), ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187)
Aug 08 04:49:09 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164548, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.131@KRBTEST.COM
Aug 08 04:49:09 dist-test-slave-04bb krb5kdc[12751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164548, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.130@KRBTEST.COM
I20260808 04:49:09.973553 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:09.959462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:45803 (local address 127.22.2.131:35187)
0808 04:49:09.959936 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:09.959959 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:09.960007 (+ 48us) server_negotiation.cc:407] Connection header received
0808 04:49:09.960750 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:09.960757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:09.961096 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:09.961414 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:09.962686 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:09.963759 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:09.965574 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:09.966036 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:09.966217 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:09.968974 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:09.968980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:09.968991 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:09.969018 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:09.970941 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:09.971522 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:09.971528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:09.971532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:09.971641 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:09.972005 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:09.972008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:09.972009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:09.972393 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:09.972584 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:09.973096 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 04:49:09.973348 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":129,"threads_started":1}
I20260808 04:49:09.973539 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:09.959216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35187 (local address 127.22.2.129:45803)
0808 04:49:09.959619 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:09.959633 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:09.959742 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:09.961457 (+ 1715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:09.961463 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:09.962040 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:09.962511 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:09.962525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:09.963940 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:09.963945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:09.965449 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:09.965460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:09.966165 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:09.966169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:09.966333 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:09.967104 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:09.967117 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:09.968728 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:09.971073 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:09.971079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:09.971093 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:09.971386 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:09.971740 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:09.971744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:09.971746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:09.971888 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:09.972516 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:09.972544 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:09.972792 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 04:49:09.972984 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 04:49:09.973351 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":161,"spinlock_wait_cycles":4992,"thread_start_us":68,"threads_started":1}
I20260808 04:49:09.973695 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:09.959410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56073 (local address 127.22.2.130:41115)
0808 04:49:09.959976 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:09.959988 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:09.960022 (+ 34us) server_negotiation.cc:407] Connection header received
0808 04:49:09.960845 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:09.960853 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:09.961165 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:09.961481 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:09.962708 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:09.963952 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:09.965586 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:09.966098 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:09.966302 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:09.969317 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:09.969322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:09.969333 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:09.969364 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:09.971082 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:09.971598 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:09.971604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:09.971608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:09.971739 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:09.972062 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:09.972066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:09.972068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:09.972453 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:09.972682 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:09.973128 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 04:49:09.973456 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":119,"threads_started":1}
I20260808 04:49:09.975234 15228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:09.959209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41115 (local address 127.22.2.129:56073)
0808 04:49:09.959573 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:09.959593 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:09.959725 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:09.961459 (+ 1734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:09.961465 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:09.962040 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:09.962512 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:09.962526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:09.964132 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:09.964139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:09.965446 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:09.965457 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:09.966250 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:09.966254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:09.966373 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:09.967039 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:09.967056 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:09.969095 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:09.971224 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:09.971227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:09.971230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:09.971463 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:09.971851 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:09.971855 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:09.971857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:09.971955 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:09.972648 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:09.972652 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:09.972807 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 04:49:09.972985 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 04:49:09.973457 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":266,"spinlock_wait_cycles":17920,"thread_start_us":87,"threads_started":1}
W20260808 04:49:09.976497 14632 leader_election.cc:341] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187): Not found: Tablet not found: 079a7773e81041e8938c9df3141ba819
W20260808 04:49:09.977442 14629 leader_election.cc:341] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115): Not found: Tablet not found: 079a7773e81041e8938c9df3141ba819
I20260808 04:49:09.977535 14629 leader_election.cc:302] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d88a10195c493da8422cccbc4765ad; no voters: 90a339c599df4a77ace7dc0e2aca2d46, ae30ca4cf9a347db807121c92b1778bc
I20260808 04:49:09.977800 15224 raft_consensus.cc:2726] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:49:10.218328 15228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:10.218004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:40443)
0808 04:49:10.218162 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:10.218238 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 04:49:10.391392 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:10.390945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:46507)
0808 04:49:10.391227 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:10.391282 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":92,"threads_started":1}
I20260808 04:49:10.575739 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:10.575328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:41439)
0808 04:49:10.575585 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:10.575639 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":74,"threads_started":1}
I20260808 04:49:11.219851 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:11.219347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:39989)
0808 04:49:11.219678 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:11.219748 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":83,"threads_started":1}
I20260808 04:49:11.392906 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:11.392385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:54405)
0808 04:49:11.392732 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:11.392804 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":88,"threads_started":1}
I20260808 04:49:11.577201 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:11.576707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:50823)
0808 04:49:11.577004 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:11.577075 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":83,"threads_started":1}
I20260808 04:49:11.870019 15242 raft_consensus.cc:482] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:11.870159 15242 raft_consensus.cc:504] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:49:11.870735 15242 leader_election.cc:288] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115), ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187)
W20260808 04:49:11.871599 14632 leader_election.cc:341] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187): Not found: Tablet not found: 079a7773e81041e8938c9df3141ba819
W20260808 04:49:11.871713 14629 leader_election.cc:341] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115): Not found: Tablet not found: 079a7773e81041e8938c9df3141ba819
I20260808 04:49:11.871834 14629 leader_election.cc:302] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d88a10195c493da8422cccbc4765ad; no voters: 90a339c599df4a77ace7dc0e2aca2d46, ae30ca4cf9a347db807121c92b1778bc
I20260808 04:49:11.872031 15242 raft_consensus.cc:2726] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:49:12.221230 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:12.220775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:47415)
0808 04:49:12.221032 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:12.221086 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260808 04:49:12.394063 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:12.393658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:42003)
0808 04:49:12.393916 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:12.393971 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260808 04:49:12.578428 15245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:12.577984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:37677)
0808 04:49:12.578274 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:12.578331 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":80,"threads_started":1}
I20260808 04:49:13.222657 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:13.222143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:48311)
0808 04:49:13.222482 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:13.222547 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":106,"threads_started":1}
I20260808 04:49:13.395373 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:13.394938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:39199)
0808 04:49:13.395217 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:13.395275 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260808 04:49:13.579782 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:13.579297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:33085)
0808 04:49:13.579604 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:13.579682 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":100,"threads_started":1}
I20260808 04:49:14.223855 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:14.223443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:44265)
0808 04:49:14.223697 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:14.223755 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260808 04:49:14.396765 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:14.396299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:41597)
0808 04:49:14.396613 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:14.396670 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
I20260808 04:49:14.480875 15253 raft_consensus.cc:482] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:14.481019 15253 raft_consensus.cc:504] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:49:14.481443 15253 leader_election.cc:288] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115), ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187)
W20260808 04:49:14.482436 14632 leader_election.cc:341] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187): Not found: Tablet not found: 079a7773e81041e8938c9df3141ba819
W20260808 04:49:14.482559 14629 leader_election.cc:341] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115): Not found: Tablet not found: 079a7773e81041e8938c9df3141ba819
I20260808 04:49:14.482635 14629 leader_election.cc:302] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d88a10195c493da8422cccbc4765ad; no voters: 90a339c599df4a77ace7dc0e2aca2d46, ae30ca4cf9a347db807121c92b1778bc
I20260808 04:49:14.482798 15253 raft_consensus.cc:2726] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:49:14.581072 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:14.580622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:48285)
0808 04:49:14.580924 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:14.580983 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":86,"threads_started":1}
I20260808 04:49:15.225288 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:15.224732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:44131)
0808 04:49:15.225044 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:15.225117 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":89,"threads_started":1}
I20260808 04:49:15.398048 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:15.397592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:49729)
0808 04:49:15.397900 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:15.397955 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":76,"threads_started":1}
I20260808 04:49:15.582540 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:15.581893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:55995)
0808 04:49:15.582367 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:15.582445 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":79,"threads_started":1}
I20260808 04:49:16.226449 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:16.226024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:40931)
0808 04:49:16.226279 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:16.226336 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260808 04:49:16.399245 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:16.398797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:52555)
0808 04:49:16.399078 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:16.399133 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":85,"threads_started":1}
I20260808 04:49:16.583886 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:16.583430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:38939)
0808 04:49:16.583716 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:16.583777 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":91,"threads_started":1}
I20260808 04:49:16.606983 15269 raft_consensus.cc:482] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:16.607179 15269 raft_consensus.cc:504] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a339c599df4a77ace7dc0e2aca2d46" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41115 } } peers { permanent_uuid: "ae30ca4cf9a347db807121c92b1778bc" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35187 } } peers { permanent_uuid: "88d88a10195c493da8422cccbc4765ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33767 } }
I20260808 04:49:16.607664 15269 leader_election.cc:288] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115), ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187)
W20260808 04:49:16.608654 14629 leader_election.cc:341] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90a339c599df4a77ace7dc0e2aca2d46 (127.22.2.130:41115): Not found: Tablet not found: 079a7773e81041e8938c9df3141ba819
W20260808 04:49:16.608771 14632 leader_election.cc:341] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae30ca4cf9a347db807121c92b1778bc (127.22.2.131:35187): Not found: Tablet not found: 079a7773e81041e8938c9df3141ba819
I20260808 04:49:16.608906 14632 leader_election.cc:302] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88d88a10195c493da8422cccbc4765ad; no voters: 90a339c599df4a77ace7dc0e2aca2d46, ae30ca4cf9a347db807121c92b1778bc
I20260808 04:49:16.609133 15269 raft_consensus.cc:2726] T 079a7773e81041e8938c9df3141ba819 P 88d88a10195c493da8422cccbc4765ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:49:17.227784 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:17.227328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:45221)
0808 04:49:17.227620 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:17.227683 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":86,"threads_started":1}
I20260808 04:49:17.400450 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:17.400004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:49551)
0808 04:49:17.400296 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:17.400352 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":95,"threads_started":1}
I20260808 04:49:17.585196 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:17.584771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:45935)
0808 04:49:17.585043 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:17.585100 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":102,"threads_started":1}
I20260808 04:49:18.229321 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:18.228808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:57749)
0808 04:49:18.229133 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:18.229217 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":80,"threads_started":1}
I20260808 04:49:18.401836 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:18.401370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:33361)
0808 04:49:18.401647 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:18.401704 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
I20260808 04:49:18.586593 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:18.586032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:45575)
0808 04:49:18.586386 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:18.586466 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":145,"threads_started":1}
I20260808 04:49:19.230643 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:19.230197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:50469)
0808 04:49:19.230476 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:19.230542 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260808 04:49:19.403019 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:19.402609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:37677)
0808 04:49:19.402857 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:19.402910 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260808 04:49:19.587911 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:19.587448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:50547)
0808 04:49:19.587736 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:19.587797 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":80,"threads_started":1}
I20260808 04:49:20.231933 15283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:20.231486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:50883)
0808 04:49:20.231774 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:20.231835 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
I20260808 04:49:20.404330 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:20.403891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:54411)
0808 04:49:20.404173 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:20.404230 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":84,"threads_started":1}
I20260808 04:49:20.589169 15286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:20.588703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:51325)
0808 04:49:20.588998 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:20.589064 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":93,"threads_started":1}
I20260808 04:49:21.233228 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:21.232775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:40639)
0808 04:49:21.233056 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:21.233119 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":78,"threads_started":1}
I20260808 04:49:21.405529 15291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:21.405102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:43667)
0808 04:49:21.405368 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:21.405428 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260808 04:49:21.590312 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:21.589917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:37645)
0808 04:49:21.590158 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:21.590215 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260808 04:49:22.234577 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:22.234080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:38783)
0808 04:49:22.234415 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:22.234477 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":93,"threads_started":1}
I20260808 04:49:22.406804 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:22.406387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:42339)
0808 04:49:22.406645 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:22.406704 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260808 04:49:22.591583 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:22.591133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:41181)
0808 04:49:22.591407 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:22.591467 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260808 04:49:23.236119 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:23.235635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.129:42549)
0808 04:49:23.235960 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:23.236019 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":96,"threads_started":1}
I20260808 04:49:23.408242 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:23.407751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.130:38397)
0808 04:49:23.408043 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:23.408101 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
I20260808 04:49:23.582862 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 14543
I20260808 04:49:23.593082 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:23.592630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41909 (local address 127.22.2.131:33345)
0808 04:49:23.592908 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:23.592981 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
I20260808 04:49:23.596426 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 14804
I20260808 04:49:23.607235 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 15004
2026-08-08T04:49:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59800 ms)
[----------] 30 tests from SecurityITest (253233 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:23 dist-test-slave-04bb krb5kdc[15306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:23 dist-test-slave-04bb krb5kdc[15306](info): set up 2 sockets
Aug 08 04:49:23 dist-test-slave-04bb krb5kdc[15306](info): commencing operation
krb5kdc: starting...
W20260808 04:49:24.859289 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:24 dist-test-slave-04bb krb5kdc[15306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:49:24.992576 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:39371
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:39371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:25.107403 15330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:25.107649 15330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:25.107689 15330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:25.111728 15330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:25.111783 15330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:25.111802 15330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:25.111819 15330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:25.111833 15330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:25.117049 15330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:39371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:39371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:25.118057 15330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:25.119087 15330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:25.128870 15336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:25.128973 15337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:25.128952 15339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:25.128945 15330 server_base.cc:1034] running on GCE node
I20260808 04:49:25.129755 15330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:25.130854 15330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:25.132050 15330 hybrid_clock.cc:648] HybridClock initialized: now 1786164565132031 us; error 32 us; skew 500 ppm
Aug 08 04:49:25 dist-test-slave-04bb krb5kdc[15306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:25.134941 15330 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:49:25.135854 15330 webserver.cc:460] Webserver started at http://127.22.2.190:44075/ using document root <none> and password file <none>
I20260808 04:49:25.136407 15330 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:25.136466 15330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:25.136701 15330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:25.138708 15330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "5af29807cfa74df680df8c714ebcae78"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "1e5c9c43e2ba41c96c58fb95e07d85ff"
server_key_iv: "1897340a4bf6912cc4dc257c239ee052"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.139216 15330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "5af29807cfa74df680df8c714ebcae78"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "1e5c9c43e2ba41c96c58fb95e07d85ff"
server_key_iv: "1897340a4bf6912cc4dc257c239ee052"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.143023 15330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 04:49:25.145866 15349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.147298 15330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:49:25.147460 15330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "5af29807cfa74df680df8c714ebcae78"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "1e5c9c43e2ba41c96c58fb95e07d85ff"
server_key_iv: "1897340a4bf6912cc4dc257c239ee052"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.147580 15330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:25.166311 15330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:25.170135 15330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:25.170310 15330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:25.177412 15330 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:39371
I20260808 04:49:25.177420 15436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:39371 every 8 connection(s)
I20260808 04:49:25.178427 15330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:25.179303 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 15330
I20260808 04:49:25.179440 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:25.179795 22538 external_mini_cluster.cc:1428] Setting key 3476b669c8906be34672d1bfca57afd5
I20260808 04:49:25.182103 15440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:25.189015 15440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78: Bootstrap starting.
Aug 08 04:49:25 dist-test-slave-04bb krb5kdc[15306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:25.191722 15440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:25.192685 15440 log.cc:824] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:25.195101 15440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78: No bootstrap required, opened a new log
I20260808 04:49:25.195865 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.181322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.22.2.190:39371)
0808 04:49:25.181781 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:25.181812 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:25.181857 (+ 45us) server_negotiation.cc:407] Connection header received
0808 04:49:25.182734 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:25.182758 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:25.183167 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:25.183527 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:25.184279 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.185859 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.186783 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.187488 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.187671 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.190423 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:25.190430 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:25.190446 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:25.190477 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:25.192612 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.193122 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.193130 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.193138 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.193264 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.193598 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.193601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.193603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.194099 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:25.194293 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:25.194570 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 04:49:25.194875 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":133,"threads_started":1}
I20260808 04:49:25.199342 15440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af29807cfa74df680df8c714ebcae78" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39371 } }
I20260808 04:49:25.199551 15440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:25.199594 15440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af29807cfa74df680df8c714ebcae78, State: Initialized, Role: FOLLOWER
I20260808 04:49:25.199959 15440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af29807cfa74df680df8c714ebcae78" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39371 } }
I20260808 04:49:25.200073 15440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:25.200112 15440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:25.200184 15440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:25.201121 15440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af29807cfa74df680df8c714ebcae78" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39371 } }
I20260808 04:49:25.201433 15440 leader_election.cc:302] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5af29807cfa74df680df8c714ebcae78; no voters:
I20260808 04:49:25.201687 15440 leader_election.cc:288] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:25.201835 15446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:25.202088 15446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [term 1 LEADER]: Becoming Leader. State: Replica: 5af29807cfa74df680df8c714ebcae78, State: Running, Role: LEADER
I20260808 04:49:25.202468 15446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af29807cfa74df680df8c714ebcae78" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39371 } }
I20260808 04:49:25.202967 15440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:25.204068 15451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af29807cfa74df680df8c714ebcae78. Latest consensus state: current_term: 1 leader_uuid: "5af29807cfa74df680df8c714ebcae78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af29807cfa74df680df8c714ebcae78" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39371 } } }
I20260808 04:49:25.204205 15451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:25.204692 15448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5af29807cfa74df680df8c714ebcae78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af29807cfa74df680df8c714ebcae78" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39371 } } }
I20260808 04:49:25.204841 15448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78 [sys.catalog]: This master's current role is: LEADER
W20260808 04:49:25.207974 15470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:49:25.208058 15470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:49:25.208189 15463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:25.209169 15463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:25.214578 15463 catalog_manager.cc:1347] Generated new cluster ID: 866fec189c424418b136cc18e08050ca
I20260808 04:49:25.214653 15463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:25.221067 15463 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:25.222050 15463 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:25.232056 15463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78: Generated new TSK 0
I20260808 04:49:25.233009 15463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:49:25.270078 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39371
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:25.390412 15480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:25.390707 15480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:25.390791 15480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:25.394892 15480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:25.394982 15480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:25.395074 15480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:49:25.400183 15480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:25.401299 15480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:25.402335 15480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:25.412348 15487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:25.412333 15490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:25.412333 15488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:25.412595 15480 server_base.cc:1034] running on GCE node
I20260808 04:49:25.413112 15480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:25.413770 15480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:25.414983 15480 hybrid_clock.cc:648] HybridClock initialized: now 1786164565414883 us; error 79 us; skew 500 ppm
Aug 08 04:49:25 dist-test-slave-04bb krb5kdc[15306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:25.417979 15480 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:49:25.418831 15480 webserver.cc:460] Webserver started at http://127.22.2.129:46031/ using document root <none> and password file <none>
I20260808 04:49:25.419404 15480 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:25.419498 15480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:25.419739 15480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:25.421857 15480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "035e644d1e414da490330c1884e6e2ad"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "5441ff295bae916bf8b1de562ab8219f"
server_key_iv: "5fae2dc0263e404bc5ae7c9766f1fe55"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.422436 15480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "035e644d1e414da490330c1884e6e2ad"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "5441ff295bae916bf8b1de562ab8219f"
server_key_iv: "5fae2dc0263e404bc5ae7c9766f1fe55"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.426287 15480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 04:49:25.428829 15498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.430068 15480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:49:25.430253 15480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "035e644d1e414da490330c1884e6e2ad"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "5441ff295bae916bf8b1de562ab8219f"
server_key_iv: "5fae2dc0263e404bc5ae7c9766f1fe55"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.430367 15480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:25.456881 15480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:25.460003 15480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:25.460220 15480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:25.460893 15480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:25.461740 15480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:25.461812 15480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.461886 15480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:25.461928 15480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.471076 15480 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:35103
I20260808 04:49:25.471110 15674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:35103 every 8 connection(s)
I20260808 04:49:25.471978 15480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:49:25.477679 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 15480
I20260808 04:49:25.477841 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:49:25.478158 22538 external_mini_cluster.cc:1428] Setting key 7e6bd5037184bb41d29bf47c00920bb5
Aug 08 04:49:25 dist-test-slave-04bb krb5kdc[15306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:25.486980 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.473853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:39069 (local address 127.22.2.190:39371)
0808 04:49:25.474013 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:25.474017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:25.474877 (+ 860us) server_negotiation.cc:407] Connection header received
0808 04:49:25.476112 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:25.476115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:25.476163 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:25.476246 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:25.477469 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.478291 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.479705 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.479957 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.480070 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.482843 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:25.482847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:25.482852 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:25.482883 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:25.484433 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.485186 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.485190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.485192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.485277 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.485700 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.485702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.485704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.485879 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:25.485956 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:25.486734 (+ 778us) server_negotiation.cc:299] Negotiation successful
0808 04:49:25.486862 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 04:49:25.488162 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.474235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:39371 (local address 127.22.2.129:39069)
0808 04:49:25.474719 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:25.474763 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:25.475908 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:25.476413 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:25.476419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:25.476841 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:25.477312 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.477325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.478432 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.478435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.479578 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.479588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.480122 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.480127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.480324 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.481020 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:25.481036 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:25.482659 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:25.484598 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:25.484608 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:25.484627 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:25.485054 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:25.485382 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:25.485388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:25.485391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:25.485585 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:25.486003 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:25.486013 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:25.486395 (+ 382us) client_negotiation.cc:769] Sending connection context
0808 04:49:25.486629 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 04:49:25.487027 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":118,"threads_started":1}
I20260808 04:49:25.489529 15677 heartbeater.cc:347] Connected to a master server at 127.22.2.190:39371
I20260808 04:49:25.489796 15677 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:49:25.490298 15677 heartbeater.cc:511] Master 127.22.2.190:39371 requested a full tablet report, sending...
I20260808 04:49:25.491983 15377 ts_manager.cc:194] Registered new tserver with Master: 035e644d1e414da490330c1884e6e2ad (127.22.2.129:35103)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 04:49:25.493235 15377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:39069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:49:25.511854 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39371
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:25.628002 15682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:25.628269 15682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:25.628346 15682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:25.632313 15682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:25.632398 15682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:25.632491 15682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:49:25.637715 15682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:25.638700 15682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:25.639689 15682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:25.649282 15689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:25.649290 15688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:25.649302 15691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:25.649686 15682 server_base.cc:1034] running on GCE node
I20260808 04:49:25.650032 15682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:25.650640 15682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:25.651800 15682 hybrid_clock.cc:648] HybridClock initialized: now 1786164565651785 us; error 27 us; skew 500 ppm
Aug 08 04:49:25 dist-test-slave-04bb krb5kdc[15306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:25.654974 15682 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:49:25.655903 15682 webserver.cc:460] Webserver started at http://127.22.2.130:42851/ using document root <none> and password file <none>
I20260808 04:49:25.656479 15682 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:25.656607 15682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:25.656847 15682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:25.658811 15682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "951389f8d4554af08f4e0cf67807c775"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "a3031ecf6dd00b4f31d4f9fd5380d64c"
server_key_iv: "deccddc3e1862f6bd0645cf29435691f"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.659381 15682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "951389f8d4554af08f4e0cf67807c775"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "a3031ecf6dd00b4f31d4f9fd5380d64c"
server_key_iv: "deccddc3e1862f6bd0645cf29435691f"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.663132 15682 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 04:49:25.665729 15701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.666970 15682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:25.667153 15682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "951389f8d4554af08f4e0cf67807c775"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "a3031ecf6dd00b4f31d4f9fd5380d64c"
server_key_iv: "deccddc3e1862f6bd0645cf29435691f"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.667271 15682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:25.683368 15682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:25.686314 15682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:25.686539 15682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:25.687131 15682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:25.687961 15682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:25.688035 15682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.688115 15682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:25.688160 15682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.697552 15682 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:34809
I20260808 04:49:25.697582 15881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:34809 every 8 connection(s)
I20260808 04:49:25.698436 15682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:49:25.698832 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 15682
I20260808 04:49:25.698956 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:49:25.699299 22538 external_mini_cluster.cc:1428] Setting key 892934e547fa21651bfed3d779aafc66
Aug 08 04:49:25 dist-test-slave-04bb krb5kdc[15306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:49:25.714167 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.700490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:47909 (local address 127.22.2.190:39371)
0808 04:49:25.700653 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:25.700658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:25.702002 (+ 1344us) server_negotiation.cc:407] Connection header received
0808 04:49:25.702975 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:25.702982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:25.703052 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:25.703164 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:25.704910 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.705590 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.707046 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.707277 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.707393 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.710188 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:25.710192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:25.710195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:25.710219 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:25.711760 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.712319 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.712324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.712325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.712416 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.712886 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.712890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.712892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.713087 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:25.713178 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:25.713843 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 04:49:25.713988 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 04:49:25.714828 15887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.701045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:39371 (local address 127.22.2.130:47909)
0808 04:49:25.701843 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:25.701885 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:25.702700 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:25.703384 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:25.703395 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:25.703929 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:25.704730 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.704754 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.705764 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.705774 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.706915 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.706926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.707433 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.707438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.707618 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.708267 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:25.708283 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:25.710001 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:25.711912 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:25.711917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:25.711928 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:25.712184 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:25.712513 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:25.712515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:25.712517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:25.712708 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:25.713200 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:25.713206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:25.713484 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 04:49:25.713764 (+ 280us) client_negotiation.cc:240] Negotiation successful
0808 04:49:25.714077 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":161,"threads_started":1}
I20260808 04:49:25.716080 15882 heartbeater.cc:347] Connected to a master server at 127.22.2.190:39371
I20260808 04:49:25.716398 15882 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:25.717053 15882 heartbeater.cc:511] Master 127.22.2.190:39371 requested a full tablet report, sending...
I20260808 04:49:25.718298 15377 ts_manager.cc:194] Registered new tserver with Master: 951389f8d4554af08f4e0cf67807c775 (127.22.2.130:34809)
I20260808 04:49:25.719177 15377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:47909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:49:25.731868 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39371
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:25.847162 15893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:25.847409 15893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:25.847451 15893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:25.851511 15893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:25.851572 15893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:25.851676 15893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:49:25.857127 15893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:39371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:25.858243 15893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:25.859301 15893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:25.869699 15902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:25.869760 15906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:25.869766 15893 server_base.cc:1034] running on GCE node
W20260808 04:49:25.869947 15903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:25.870440 15893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:25.871088 15893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:25.872308 15893 hybrid_clock.cc:648] HybridClock initialized: now 1786164565872282 us; error 35 us; skew 500 ppm
Aug 08 04:49:25 dist-test-slave-04bb krb5kdc[15306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:25.875404 15893 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:49:25.876374 15893 webserver.cc:460] Webserver started at http://127.22.2.131:33889/ using document root <none> and password file <none>
I20260808 04:49:25.877034 15893 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:25.877139 15893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:25.877390 15893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:25.879482 15893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "eb050a9662d343e9afb1bf74b3a3ef12"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "ec2726119bd5ba32d58e570abb1e15a0"
server_key_iv: "78f428434f4f5161072de6bdd02877ad"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.880069 15893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "eb050a9662d343e9afb1bf74b3a3ef12"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "ec2726119bd5ba32d58e570abb1e15a0"
server_key_iv: "78f428434f4f5161072de6bdd02877ad"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.884070 15893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 04:49:25.886973 15914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.888433 15893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 04:49:25.888616 15893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "eb050a9662d343e9afb1bf74b3a3ef12"
format_stamp: "Formatted at 2026-08-08 04:49:25 on dist-test-slave-04bb"
server_key: "ec2726119bd5ba32d58e570abb1e15a0"
server_key_iv: "78f428434f4f5161072de6bdd02877ad"
server_key_version: "encryptionkey@0"
I20260808 04:49:25.888747 15893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:25.904265 15893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:25.907538 15893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:25.907760 15893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:25.908409 15893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:25.909365 15893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:25.909443 15893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.909518 15893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:25.909575 15893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:25.919320 15893 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:43379
I20260808 04:49:25.919371 16083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:43379 every 8 connection(s)
I20260808 04:49:25.920327 15893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:49:25.929235 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 15893
I20260808 04:49:25.929411 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:49:25.929721 22538 external_mini_cluster.cc:1428] Setting key c60d0c3bb1ff9018ffa47d2091343f8a
Aug 08 04:49:25 dist-test-slave-04bb krb5kdc[15306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:25.935298 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.922447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:39547 (local address 127.22.2.190:39371)
0808 04:49:25.922611 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:25.922615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:25.923728 (+ 1113us) server_negotiation.cc:407] Connection header received
0808 04:49:25.924701 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:25.924704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:25.924762 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:25.924846 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:25.926043 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.926650 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.927948 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.928173 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.928267 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.931525 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:25.931529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:25.931531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:25.931555 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:25.933192 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.933678 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.933681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.933682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.933756 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.934199 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.934203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.934204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.934411 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:25.934508 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:25.935059 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 04:49:25.935175 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 04:49:25.936116 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.922878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:39371 (local address 127.22.2.131:39547)
0808 04:49:25.923566 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:25.923612 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:25.924400 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:25.925045 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:25.925052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:25.925401 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:25.925900 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.925914 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.926806 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.926810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.927806 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.927818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.928359 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.928363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.928607 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.929319 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:25.929338 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:25.931347 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:25.933303 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:25.933308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:25.933321 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:25.933587 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:25.933856 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:25.933859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:25.933860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:25.934021 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:25.934529 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:25.934535 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:25.934784 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 04:49:25.934965 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 04:49:25.935224 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":124,"threads_started":1}
I20260808 04:49:25.937325 16084 heartbeater.cc:347] Connected to a master server at 127.22.2.190:39371
I20260808 04:49:25.937587 16084 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:25.938130 16084 heartbeater.cc:511] Master 127.22.2.190:39371 requested a full tablet report, sending...
I20260808 04:49:25.939262 15377 ts_manager.cc:194] Registered new tserver with Master: eb050a9662d343e9afb1bf74b3a3ef12 (127.22.2.131:43379)
I20260808 04:49:25.939833 15377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:39547
I20260808 04:49:25.943776 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:49:25.954048 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.946052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50672 (local address 127.22.2.190:39371)
0808 04:49:25.946270 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:25.946274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:25.946403 (+ 129us) server_negotiation.cc:407] Connection header received
0808 04:49:25.946506 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:25.946509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:25.946548 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:25.946620 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:25.947366 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.947915 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.948885 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.949111 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.949250 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.950656 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:25.950660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:25.950662 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:25.950686 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:25.952300 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.952822 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.952825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.952827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.952910 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:25.953208 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:25.953211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:25.953212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:25.953426 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:25.953538 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:25.953763 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 04:49:25.953855 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 04:49:25.957068 15377 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:49:25.960317 15377 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:49:25.974929 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.969036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35103 (local address 127.0.0.1:40546)
0808 04:49:25.969912 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:25.969925 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:25.970009 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:25.970549 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:25.970555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:25.970581 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:25.970706 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.970713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.972199 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.972202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.973476 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.973486 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.974471 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.974473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.974557 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.974590 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 04:49:25.974693 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 04:49:25.974779 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":10,"thread_start_us":94,"threads_started":1}
I20260808 04:49:25.974992 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.969506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.22.2.129:35103)
0808 04:49:25.970022 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:25.970040 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:25.970067 (+ 27us) server_negotiation.cc:407] Connection header received
0808 04:49:25.970149 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:25.970154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:25.970372 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:25.970546 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:25.970834 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.972064 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.973602 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.974344 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.974461 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.974608 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:25.974739 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 04:49:25.974843 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":75,"threads_started":1}
I20260808 04:49:25.975324 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.968690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:43379 (local address 127.0.0.1:36702)
0808 04:49:25.969509 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:25.969558 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:25.969684 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:25.970806 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:25.970810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:25.970830 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:25.970943 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.970950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.972351 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.972354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.973847 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.973859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.974919 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.974923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.975022 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.975059 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 04:49:25.975150 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 04:49:25.975208 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":71,"thread_start_us":111,"threads_started":1}
I20260808 04:49:25.975672 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.968843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.22.2.131:43379)
0808 04:49:25.970195 (+ 1352us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:25.970206 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:25.970226 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:49:25.970290 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:25.970294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:25.970448 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:25.970578 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:25.971073 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.972188 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.973998 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.974796 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.974902 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.975032 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:25.975417 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 04:49:25.975521 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1246,"thread_start_us":131,"threads_started":1}
I20260808 04:49:25.976500 16110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.968739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:34809 (local address 127.0.0.1:36594)
0808 04:49:25.969837 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:25.969852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:25.969947 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:25.970873 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:25.970876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:25.970893 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:25.970982 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.970988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.972071 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.972077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.973847 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:25.973859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.976132 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.976135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:25.976201 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.976218 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 04:49:25.976309 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 04:49:25.976376 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"mutex_wait_us":230,"thread_start_us":98,"threads_started":1}
I20260808 04:49:25.976594 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:25.969370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36594 (local address 127.22.2.130:34809)
0808 04:49:25.970422 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:25.970432 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:25.970452 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:49:25.970554 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:25.970559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:25.970754 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:25.970909 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:25.971083 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.971946 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.975117 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:25.975816 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:25.975925 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:25.976075 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:25.976341 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 04:49:25.976424 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":99,"threads_started":1}
I20260808 04:49:25.978711 15988 tablet_service.cc:1467] Processing CreateTablet for tablet 693b7e5b6b4541f2a759e0cf059b41c3 (DEFAULT_TABLE table=test-table [id=bef95b851f5444f48c9ac005acec05dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:25.978711 15576 tablet_service.cc:1467] Processing CreateTablet for tablet 693b7e5b6b4541f2a759e0cf059b41c3 (DEFAULT_TABLE table=test-table [id=bef95b851f5444f48c9ac005acec05dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:25.979187 15777 tablet_service.cc:1467] Processing CreateTablet for tablet 693b7e5b6b4541f2a759e0cf059b41c3 (DEFAULT_TABLE table=test-table [id=bef95b851f5444f48c9ac005acec05dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:25.979892 15576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 693b7e5b6b4541f2a759e0cf059b41c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:25.979902 15988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 693b7e5b6b4541f2a759e0cf059b41c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:25.980180 15777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 693b7e5b6b4541f2a759e0cf059b41c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:25.986850 16113 tablet_bootstrap.cc:492] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Bootstrap starting.
I20260808 04:49:25.987512 16114 tablet_bootstrap.cc:492] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Bootstrap starting.
I20260808 04:49:25.988855 16116 tablet_bootstrap.cc:492] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad: Bootstrap starting.
I20260808 04:49:25.989926 16114 tablet_bootstrap.cc:654] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:25.990432 16113 tablet_bootstrap.cc:654] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:25.990711 16116 tablet_bootstrap.cc:654] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:25.991261 16113 log.cc:824] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:25.991261 16114 log.cc:824] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:25.991596 16116 log.cc:824] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:25.993283 16114 tablet_bootstrap.cc:492] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: No bootstrap required, opened a new log
I20260808 04:49:25.993348 16113 tablet_bootstrap.cc:492] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: No bootstrap required, opened a new log
I20260808 04:49:25.993348 16116 tablet_bootstrap.cc:492] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad: No bootstrap required, opened a new log
I20260808 04:49:25.993489 16114 ts_tablet_manager.cc:1397] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260808 04:49:25.993528 16116 ts_tablet_manager.cc:1397] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 04:49:25.993528 16113 ts_tablet_manager.cc:1397] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 04:49:25.996276 16114 raft_consensus.cc:348] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:25.996479 16114 raft_consensus.cc:374] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:25.996562 16114 raft_consensus.cc:729] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb050a9662d343e9afb1bf74b3a3ef12, State: Initialized, Role: FOLLOWER
I20260808 04:49:25.997032 16114 consensus_queue.cc:260] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:25.997380 16113 raft_consensus.cc:348] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:25.997684 16084 heartbeater.cc:503] Master 127.22.2.190:39371 was elected leader, sending a full tablet report...
I20260808 04:49:25.997699 16113 raft_consensus.cc:374] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:25.997766 16113 raft_consensus.cc:729] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 951389f8d4554af08f4e0cf67807c775, State: Initialized, Role: FOLLOWER
I20260808 04:49:25.998067 16114 ts_tablet_manager.cc:1428] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 04:49:25.998262 16113 consensus_queue.cc:260] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:25.997700 16116 raft_consensus.cc:348] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:25.998536 16116 raft_consensus.cc:374] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:25.998627 16116 raft_consensus.cc:729] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035e644d1e414da490330c1884e6e2ad, State: Initialized, Role: FOLLOWER
I20260808 04:49:25.999027 15882 heartbeater.cc:503] Master 127.22.2.190:39371 was elected leader, sending a full tablet report...
I20260808 04:49:25.999082 16116 consensus_queue.cc:260] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:25.999354 16113 ts_tablet_manager.cc:1428] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:49:25.999859 15677 heartbeater.cc:503] Master 127.22.2.190:39371 was elected leader, sending a full tablet report...
I20260808 04:49:25.999998 16116 ts_tablet_manager.cc:1428] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:49:26.167323 16127 raft_consensus.cc:482] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:26.167591 16127 raft_consensus.cc:504] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:26.168898 16127 leader_election.cc:288] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb050a9662d343e9afb1bf74b3a3ef12 (127.22.2.131:43379), 951389f8d4554af08f4e0cf67807c775 (127.22.2.130:34809)
W20260808 04:49:26.171401 16085 tablet.cc:2369] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:26.173547 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.169286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:41669 (local address 127.22.2.131:43379)
0808 04:49:26.169427 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:26.169432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:26.169675 (+ 243us) server_negotiation.cc:407] Connection header received
0808 04:49:26.169834 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:26.169837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:26.169893 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:26.169976 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:26.170408 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.171208 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.172518 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.172960 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.173107 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:26.173164 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:26.173355 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 04:49:26.173426 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 04:49:26.173573 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.169134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:43379 (local address 127.22.2.129:41669)
0808 04:49:26.169571 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:26.169587 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:26.169725 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:26.170119 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:26.170123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:26.170149 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:26.170280 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:26.170286 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.171386 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.171389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:26.172389 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:26.172399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.173107 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.173111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:26.173216 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:26.173242 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 04:49:26.173336 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 04:49:26.173411 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":4,"thread_start_us":144,"threads_started":1}
I20260808 04:49:26.173787 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.169141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:34809 (local address 127.22.2.129:54031)
0808 04:49:26.169621 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:26.169634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:26.169733 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:26.170081 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:26.170085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:26.170121 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:26.170289 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:26.170294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.171342 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.171345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:26.172430 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:26.172440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.173119 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.173122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:26.173449 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:26.173469 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 04:49:26.173570 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 04:49:26.173639 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":61,"thread_start_us":147,"threads_started":1}
I20260808 04:49:26.173774 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.169257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:54031 (local address 127.22.2.130:34809)
0808 04:49:26.169390 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:26.169394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:26.169730 (+ 336us) server_negotiation.cc:407] Connection header received
0808 04:49:26.169834 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:26.169837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:26.169894 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:26.169977 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:26.170459 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.171228 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.172600 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.172990 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.173119 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:26.173171 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:26.173588 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 04:49:26.173662 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:49:26.174338 16019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "693b7e5b6b4541f2a759e0cf059b41c3" candidate_uuid: "035e644d1e414da490330c1884e6e2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" is_pre_election: true
I20260808 04:49:26.174547 15806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "693b7e5b6b4541f2a759e0cf059b41c3" candidate_uuid: "035e644d1e414da490330c1884e6e2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "951389f8d4554af08f4e0cf67807c775" is_pre_election: true
I20260808 04:49:26.174646 16019 raft_consensus.cc:2454] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 035e644d1e414da490330c1884e6e2ad in term 0.
I20260808 04:49:26.174883 15806 raft_consensus.cc:2454] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 035e644d1e414da490330c1884e6e2ad in term 0.
I20260808 04:49:26.175264 15499 leader_election.cc:302] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 035e644d1e414da490330c1884e6e2ad, eb050a9662d343e9afb1bf74b3a3ef12; no voters:
I20260808 04:49:26.175580 16127 raft_consensus.cc:2781] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:49:26.175661 16127 raft_consensus.cc:482] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:49:26.175693 16127 raft_consensus.cc:3037] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:26.176811 16127 raft_consensus.cc:504] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:26.177345 16127 leader_election.cc:288] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [CANDIDATE]: Term 1 election: Requested vote from peers eb050a9662d343e9afb1bf74b3a3ef12 (127.22.2.131:43379), 951389f8d4554af08f4e0cf67807c775 (127.22.2.130:34809)
I20260808 04:49:26.177556 15806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "693b7e5b6b4541f2a759e0cf059b41c3" candidate_uuid: "035e644d1e414da490330c1884e6e2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "951389f8d4554af08f4e0cf67807c775"
I20260808 04:49:26.177594 16019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "693b7e5b6b4541f2a759e0cf059b41c3" candidate_uuid: "035e644d1e414da490330c1884e6e2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb050a9662d343e9afb1bf74b3a3ef12"
I20260808 04:49:26.177706 16019 raft_consensus.cc:3037] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:26.177707 15806 raft_consensus.cc:3037] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:26.178792 15806 raft_consensus.cc:2454] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 035e644d1e414da490330c1884e6e2ad in term 1.
I20260808 04:49:26.178946 16019 raft_consensus.cc:2454] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 035e644d1e414da490330c1884e6e2ad in term 1.
I20260808 04:49:26.179205 15505 leader_election.cc:302] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 035e644d1e414da490330c1884e6e2ad, 951389f8d4554af08f4e0cf67807c775; no voters:
I20260808 04:49:26.179400 16127 raft_consensus.cc:2781] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:26.179625 16127 raft_consensus.cc:686] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [term 1 LEADER]: Becoming Leader. State: Replica: 035e644d1e414da490330c1884e6e2ad, State: Running, Role: LEADER
I20260808 04:49:26.180001 16127 consensus_queue.cc:237] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } }
I20260808 04:49:26.183207 15375 catalog_manager.cc:5495] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 035e644d1e414da490330c1884e6e2ad (127.22.2.129). New cstate: current_term: 1 leader_uuid: "035e644d1e414da490330c1884e6e2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "035e644d1e414da490330c1884e6e2ad" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 35103 } health_report { overall_health: HEALTHY } } }
W20260808 04:49:26.199695 15884 tablet.cc:2369] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:26.211601 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.207719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.22.2.129:35103)
0808 04:49:26.207867 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:26.207871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:26.207885 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:49:26.208045 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:26.208048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:26.208105 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:26.208198 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:49:26.208504 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.209027 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.210016 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.210228 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.210377 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:26.210698 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:49:26.211108 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:49:26.211196 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:26.211360 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 04:49:26.211459 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:49:26.217923 15806 raft_consensus.cc:1261] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 1 FOLLOWER]: Refusing update from remote peer 035e644d1e414da490330c1884e6e2ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:26.217934 16019 raft_consensus.cc:1261] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 1 FOLLOWER]: Refusing update from remote peer 035e644d1e414da490330c1884e6e2ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:26.218698 16130 consensus_queue.cc:1035] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "951389f8d4554af08f4e0cf67807c775" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:26.218912 16127 consensus_queue.cc:1035] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb050a9662d343e9afb1bf74b3a3ef12" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 04:49:26.223289 15678 tablet.cc:2369] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:26.227435 16139 mvcc.cc:205] Tried to move back new op lower bound from 7316130063221039104 to 7316130063076470784. Current Snapshot: MvccSnapshot[applied={T|T < 7316130063221039104}]
I20260808 04:49:26.233834 15377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:26.234000 15377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:26.236413 15377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5af29807cfa74df680df8c714ebcae78: Sending DeleteTablet for 3 replicas of tablet 693b7e5b6b4541f2a759e0cf059b41c3
I20260808 04:49:26.237175 15988 tablet_service.cc:1514] Processing DeleteTablet for tablet 693b7e5b6b4541f2a759e0cf059b41c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:26 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:36702
I20260808 04:49:26.237181 15777 tablet_service.cc:1514] Processing DeleteTablet for tablet 693b7e5b6b4541f2a759e0cf059b41c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:26 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:36594
I20260808 04:49:26.237368 15576 tablet_service.cc:1514] Processing DeleteTablet for tablet 693b7e5b6b4541f2a759e0cf059b41c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:26 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40546
I20260808 04:49:26.237972 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 15480
I20260808 04:49:26.238009 16153 tablet_replica.cc:330] T 693b7e5b6b4541f2a759e0cf059b41c3 P 035e644d1e414da490330c1884e6e2ad: stopping tablet replica
I20260808 04:49:26.238245 16149 tablet_replica.cc:330] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: stopping tablet replica
I20260808 04:49:26.238482 16152 tablet_replica.cc:330] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: stopping tablet replica
I20260808 04:49:26.238829 16152 raft_consensus.cc:2229] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:26.239197 16149 raft_consensus.cc:2229] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:26.241034 16152 raft_consensus.cc:2258] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:26.241400 16149 raft_consensus.cc:2258] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:26.244235 16152 ts_tablet_manager.cc:1905] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:26.247083 16149 ts_tablet_manager.cc:1905] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:26.247277 16152 ts_tablet_manager.cc:1918] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:26.247375 16152 log.cc:1196] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/693b7e5b6b4541f2a759e0cf059b41c3
I20260808 04:49:26.247705 16152 ts_tablet_manager.cc:1939] T 693b7e5b6b4541f2a759e0cf059b41c3 P 951389f8d4554af08f4e0cf67807c775: Deleting consensus metadata
I20260808 04:49:26.249123 15353 catalog_manager.cc:4841] TS 951389f8d4554af08f4e0cf67807c775 (127.22.2.130:34809): tablet 693b7e5b6b4541f2a759e0cf059b41c3 (table test-table [id=bef95b851f5444f48c9ac005acec05dd]) successfully deleted
I20260808 04:49:26.249830 16149 ts_tablet_manager.cc:1918] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:26.249904 16149 log.cc:1196] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/693b7e5b6b4541f2a759e0cf059b41c3
I20260808 04:49:26.250217 16149 ts_tablet_manager.cc:1939] T 693b7e5b6b4541f2a759e0cf059b41c3 P eb050a9662d343e9afb1bf74b3a3ef12: Deleting consensus metadata
I20260808 04:49:26.251262 15351 catalog_manager.cc:4841] TS eb050a9662d343e9afb1bf74b3a3ef12 (127.22.2.131:43379): tablet 693b7e5b6b4541f2a759e0cf059b41c3 (table test-table [id=bef95b851f5444f48c9ac005acec05dd]) successfully deleted
W20260808 04:49:26.253321 15356 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:35103 (error 108)
I20260808 04:49:26.253507 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 15682
I20260808 04:49:26.261912 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.261494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:35103 (local address 127.0.0.1:40566)
0808 04:49:26.261680 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:26.261786 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 04:49:26.262192 15356 catalog_manager.cc:4584] TS 035e644d1e414da490330c1884e6e2ad (127.22.2.129:35103): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 693b7e5b6b4541f2a759e0cf059b41c3: Network error: Client connection negotiation failed: client connection to 127.22.2.129:35103: connect: Connection refused (error 111)
I20260808 04:49:26.268149 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 15893
I20260808 04:49:26.276032 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 15330
2026-08-08T04:49:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2670 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T04:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:26Z Disabled control of system clock
I20260808 04:49:26.316089 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41025
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 04:49:26.429329 16160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:26.429575 16160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:26.429610 16160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:26.433590 16160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:26.433641 16160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:26.433656 16160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:26.433671 16160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:26.433686 16160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:26.438237 16160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:26.439174 16160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:26.440466 16160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:26.449064 16171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:26.449110 16172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:26.449143 16175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:26.449303 16160 server_base.cc:1034] running on GCE node
I20260808 04:49:26.449947 16160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:26.451032 16160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:26.452230 16160 hybrid_clock.cc:648] HybridClock initialized: now 1786164566452203 us; error 43 us; skew 500 ppm
I20260808 04:49:26.454157 16160 webserver.cc:460] Webserver started at http://127.22.2.190:39137/ using document root <none> and password file <none>
I20260808 04:49:26.454778 16160 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:26.454871 16160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:26.455106 16160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:26.457243 16160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "fcfa74945a6f428b9482c4c4c9090095"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "38df6b65a69bc7240cff64550012359a"
server_key_iv: "6364eb7867c06087013358d595ff0e30"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.457803 16160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "fcfa74945a6f428b9482c4c4c9090095"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "38df6b65a69bc7240cff64550012359a"
server_key_iv: "6364eb7867c06087013358d595ff0e30"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.461529 16160 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:49:26.463882 16182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:26.465199 16160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:49:26.465390 16160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "fcfa74945a6f428b9482c4c4c9090095"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "38df6b65a69bc7240cff64550012359a"
server_key_iv: "6364eb7867c06087013358d595ff0e30"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.465502 16160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:26.483892 16160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:26.484583 16160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:26.484784 16160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:26.491920 16160 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41025
I20260808 04:49:26.491931 16258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41025 every 8 connection(s)
I20260808 04:49:26.492921 16160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:26.496055 16260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:26.501529 16260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095: Bootstrap starting.
I20260808 04:49:26.502063 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 16160
I20260808 04:49:26.502204 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:26.502492 22538 external_mini_cluster.cc:1428] Setting key 12f5414f8cb1ed0e26d54e7f2a381fb0
I20260808 04:49:26.504561 16260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:26.505317 16260 log.cc:824] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:26.507139 16260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095: No bootstrap required, opened a new log
I20260808 04:49:26.510812 16260 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfa74945a6f428b9482c4c4c9090095" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41025 } }
I20260808 04:49:26.511050 16260 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:26.511158 16260 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcfa74945a6f428b9482c4c4c9090095, State: Initialized, Role: FOLLOWER
I20260808 04:49:26.511658 16260 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfa74945a6f428b9482c4c4c9090095" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41025 } }
I20260808 04:49:26.511816 16260 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:26.511905 16260 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:26.512008 16260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:26.512140 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.503973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.22.2.190:41025)
0808 04:49:26.504733 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:26.504763 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:26.504801 (+ 38us) server_negotiation.cc:407] Connection header received
0808 04:49:26.505438 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:26.505466 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:26.505924 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:26.506252 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:26.506676 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.507716 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.508702 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.509175 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.509365 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:26.510311 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:26.510317 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:26.510330 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:26.510696 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:26.510758 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:26.510778 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:26.510885 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:26.511042 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 04:49:26.511319 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":149,"threads_started":1}
I20260808 04:49:26.513089 16260 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfa74945a6f428b9482c4c4c9090095" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41025 } }
I20260808 04:49:26.513435 16260 leader_election.cc:302] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcfa74945a6f428b9482c4c4c9090095; no voters:
I20260808 04:49:26.513691 16260 leader_election.cc:288] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:26.513795 16270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:26.514031 16270 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [term 1 LEADER]: Becoming Leader. State: Replica: fcfa74945a6f428b9482c4c4c9090095, State: Running, Role: LEADER
I20260808 04:49:26.514369 16270 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfa74945a6f428b9482c4c4c9090095" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41025 } }
I20260808 04:49:26.514954 16260 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:26.516222 16271 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcfa74945a6f428b9482c4c4c9090095" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfa74945a6f428b9482c4c4c9090095" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41025 } } }
I20260808 04:49:26.516232 16272 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcfa74945a6f428b9482c4c4c9090095. Latest consensus state: current_term: 1 leader_uuid: "fcfa74945a6f428b9482c4c4c9090095" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfa74945a6f428b9482c4c4c9090095" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41025 } } }
I20260808 04:49:26.516407 16271 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:26.516450 16272 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:26.516839 16281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:26.520190 16281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:26.525688 16281 catalog_manager.cc:1347] Generated new cluster ID: cc47557e361d4b7985f40c8c44966ef0
I20260808 04:49:26.525759 16281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:26.551755 16281 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:26.552975 16281 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:26.557263 16281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095: Generated new TSK 0
I20260808 04:49:26.557977 16281 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:26.566059 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41025
--builtin_ntp_servers=127.22.2.148:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 04:49:26.685374 16297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:26.685608 16297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:26.685652 16297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:26.689600 16297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:26.689657 16297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:26.689708 16297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:49:26.694401 16297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:26.695427 16297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:26.696808 16297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:26.706480 16306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:26.706534 16303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:26.706535 16304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:26.706822 16297 server_base.cc:1034] running on GCE node
I20260808 04:49:26.707221 16297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:26.707906 16297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:26.709129 16297 hybrid_clock.cc:648] HybridClock initialized: now 1786164566709076 us; error 62 us; skew 500 ppm
I20260808 04:49:26.711109 16297 webserver.cc:460] Webserver started at http://127.22.2.129:45761/ using document root <none> and password file <none>
I20260808 04:49:26.711733 16297 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:26.711829 16297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:26.712075 16297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:26.714190 16297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "2f999c6fe040463480842a845f5c849c"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "d126f47207ef3168c7fbedc0d33c636d"
server_key_iv: "153773468befbaa603397f5e79df22a2"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.714758 16297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "2f999c6fe040463480842a845f5c849c"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "d126f47207ef3168c7fbedc0d33c636d"
server_key_iv: "153773468befbaa603397f5e79df22a2"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.718752 16297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:49:26.721413 16316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:26.722903 16297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:49:26.723071 16297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "2f999c6fe040463480842a845f5c849c"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "d126f47207ef3168c7fbedc0d33c636d"
server_key_iv: "153773468befbaa603397f5e79df22a2"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.723219 16297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:26.754850 16297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:26.755632 16297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:26.755862 16297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:26.756587 16297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:26.757649 16297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:26.757730 16297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:26.757807 16297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:26.757863 16297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:26.768580 16297 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:38787
I20260808 04:49:26.768620 16490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:38787 every 8 connection(s)
I20260808 04:49:26.769685 16297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:49:26.774952 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 16297
I20260808 04:49:26.775049 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:49:26.775282 22538 external_mini_cluster.cc:1428] Setting key fb0cde582dc51b42edd1c7eaf9164947
I20260808 04:49:26.777442 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41025
--builtin_ntp_servers=127.22.2.148:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 04:49:26.781199 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.771824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:50127 (local address 127.22.2.190:41025)
0808 04:49:26.772003 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:26.772008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:26.773316 (+ 1308us) server_negotiation.cc:407] Connection header received
0808 04:49:26.774611 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:26.774614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:26.774664 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:26.774745 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:26.776578 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.777312 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.778770 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.779165 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.779295 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:26.780670 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:26.780673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:26.780675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:26.780687 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:26.780713 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:26.780721 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:26.780783 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:26.780977 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 04:49:26.781038 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 04:49:26.782385 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:26.772325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41025 (local address 127.22.2.129:50127)
0808 04:49:26.773055 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:26.773097 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:26.774402 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:26.775821 (+ 1419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:26.775828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:26.775915 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:26.776407 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:26.776419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.777479 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.777482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:26.778618 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:26.778631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:26.779365 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:26.779372 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:26.779605 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:26.780313 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:26.780318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:26.780389 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:26.780482 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:26.780816 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:26.780828 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:26.780837 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 04:49:26.780973 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 04:49:26.781293 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":162,"threads_started":1}
I20260808 04:49:26.783690 16491 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41025
I20260808 04:49:26.783937 16491 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:26.784490 16491 heartbeater.cc:511] Master 127.22.2.190:41025 requested a full tablet report, sending...
I20260808 04:49:26.786079 16207 ts_manager.cc:194] Registered new tserver with Master: 2f999c6fe040463480842a845f5c849c (127.22.2.129:38787)
I20260808 04:49:26.787210 16207 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.129:50127
W20260808 04:49:26.916465 16497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:26.916872 16497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:26.916960 16497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:26.926602 16497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:26.926755 16497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:26.926905 16497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:49:26.938122 16497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:26.939709 16497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:26.941638 16497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:26.952490 16504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:26.952510 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:26.952491 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:26.952955 16497 server_base.cc:1034] running on GCE node
I20260808 04:49:26.953502 16497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:26.954365 16497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:26.955569 16497 hybrid_clock.cc:648] HybridClock initialized: now 1786164566955543 us; error 36 us; skew 500 ppm
I20260808 04:49:26.958087 16497 webserver.cc:460] Webserver started at http://127.22.2.130:33025/ using document root <none> and password file <none>
I20260808 04:49:26.958904 16497 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:26.959002 16497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:26.959264 16497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:26.962348 16497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "629a0ce3882c4df388cc6dd7f0deff2a"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "cef3aae01c589b3b340700cf87c1d9a1"
server_key_iv: "83e56392aa544124e7cab35256a132ee"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.963163 16497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "629a0ce3882c4df388cc6dd7f0deff2a"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "cef3aae01c589b3b340700cf87c1d9a1"
server_key_iv: "83e56392aa544124e7cab35256a132ee"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.967476 16497 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 04:49:26.970014 16517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:26.971304 16497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:26.971396 16497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "629a0ce3882c4df388cc6dd7f0deff2a"
format_stamp: "Formatted at 2026-08-08 04:49:26 on dist-test-slave-04bb"
server_key: "cef3aae01c589b3b340700cf87c1d9a1"
server_key_iv: "83e56392aa544124e7cab35256a132ee"
server_key_version: "encryptionkey@0"
I20260808 04:49:26.971527 16497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:26.987552 16497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:26.988258 16497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:26.988459 16497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:26.989113 16497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:26.990078 16497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:26.990165 16497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:26.990263 16497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:26.990316 16497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:27.001294 16497 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:39201
I20260808 04:49:27.001320 16683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:39201 every 8 connection(s)
I20260808 04:49:27.002267 16497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:49:27.006368 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 16497
I20260808 04:49:27.006552 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:49:27.006973 22538 external_mini_cluster.cc:1428] Setting key e4d980ca3672b1111e2d2ae5adebf38b
I20260808 04:49:27.009492 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41025
--builtin_ntp_servers=127.22.2.148:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 04:49:27.013537 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.004621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:40639 (local address 127.22.2.190:41025)
0808 04:49:27.004809 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.004817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.005867 (+ 1050us) server_negotiation.cc:407] Connection header received
0808 04:49:27.007517 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.007520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.007581 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.007668 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:27.008915 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.009693 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.011497 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.011718 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.011777 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.013015 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:27.013019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:27.013021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:27.013034 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:27.013053 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:27.013069 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:27.013148 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.013332 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.013393 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 04:49:27.014428 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.004993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41025 (local address 127.22.2.130:40639)
0808 04:49:27.005702 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:27.005754 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:27.007290 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:27.008027 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:27.008033 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:27.008116 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:27.008715 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.008733 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.009866 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.009872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.011336 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.011352 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.011928 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.011931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.012066 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.012751 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:27.012752 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:27.012791 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:27.012853 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:27.013162 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:27.013175 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:27.013186 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 04:49:27.013314 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 04:49:27.013580 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":142,"threads_started":1}
I20260808 04:49:27.015885 16685 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41025
I20260808 04:49:27.016160 16685 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:27.016841 16685 heartbeater.cc:511] Master 127.22.2.190:41025 requested a full tablet report, sending...
I20260808 04:49:27.017954 16207 ts_manager.cc:194] Registered new tserver with Master: 629a0ce3882c4df388cc6dd7f0deff2a (127.22.2.130:39201)
I20260808 04:49:27.018536 16207 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.130:40639
W20260808 04:49:27.163661 16689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:27.163933 16689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:27.164005 16689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:27.168118 16689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:27.168191 16689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:27.168255 16689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:49:27.173345 16689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:27.174340 16689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:27.175874 16689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:27.185683 16703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:27.185747 16701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:27.185778 16700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:27.186071 16689 server_base.cc:1034] running on GCE node
I20260808 04:49:27.186491 16689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:27.187140 16689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:27.188336 16689 hybrid_clock.cc:648] HybridClock initialized: now 1786164567188320 us; error 31 us; skew 500 ppm
I20260808 04:49:27.190161 16689 webserver.cc:460] Webserver started at http://127.22.2.131:41433/ using document root <none> and password file <none>
I20260808 04:49:27.190742 16689 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:27.190833 16689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:27.191053 16689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:27.193176 16689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "f4bf2ef272784ce5bbf8b28a14ce2504"
format_stamp: "Formatted at 2026-08-08 04:49:27 on dist-test-slave-04bb"
server_key: "1ae3feb86ee8f42cd6f5c7b30003774d"
server_key_iv: "c0f04884f209663a0aa504bb0fdfb1ec"
server_key_version: "encryptionkey@0"
I20260808 04:49:27.193750 16689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "f4bf2ef272784ce5bbf8b28a14ce2504"
format_stamp: "Formatted at 2026-08-08 04:49:27 on dist-test-slave-04bb"
server_key: "1ae3feb86ee8f42cd6f5c7b30003774d"
server_key_iv: "c0f04884f209663a0aa504bb0fdfb1ec"
server_key_version: "encryptionkey@0"
I20260808 04:49:27.197439 16689 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:49:27.199928 16714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:27.201220 16689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:27.201357 16689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "f4bf2ef272784ce5bbf8b28a14ce2504"
format_stamp: "Formatted at 2026-08-08 04:49:27 on dist-test-slave-04bb"
server_key: "1ae3feb86ee8f42cd6f5c7b30003774d"
server_key_iv: "c0f04884f209663a0aa504bb0fdfb1ec"
server_key_version: "encryptionkey@0"
I20260808 04:49:27.201469 16689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:27.225754 16689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:27.226475 16689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:27.226732 16689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:27.227365 16689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:27.228473 16689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:27.228538 16689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:27.228580 16689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:27.228596 16689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:27.237883 16689 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:34617
I20260808 04:49:27.237987 16894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:34617 every 8 connection(s)
I20260808 04:49:27.239087 16689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:49:27.248369 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.240986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:41065 (local address 127.22.2.190:41025)
0808 04:49:27.241143 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.241148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.242276 (+ 1128us) server_negotiation.cc:407] Connection header received
0808 04:49:27.243603 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.243606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.243674 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.243750 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:27.244697 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.245313 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.246488 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.246732 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.246832 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.247873 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:27.247876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:27.247878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:27.247891 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:27.247908 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:27.247916 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:27.247980 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.248206 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.248261 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:49:27.248802 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 16689
I20260808 04:49:27.248924 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:49:27.249217 22538 external_mini_cluster.cc:1428] Setting key 30c9d49244c2de06fcdfed992a295d67
I20260808 04:49:27.249210 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.241427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41025 (local address 127.22.2.131:41065)
0808 04:49:27.242112 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:27.242154 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:27.243396 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:27.243910 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:27.243916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:27.244013 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:27.244479 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.244491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.245447 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.245449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.246369 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.246380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.246878 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.246881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.247018 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.247607 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:27.247608 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:27.247652 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:27.247714 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:27.247996 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:27.248003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:27.248008 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:49:27.248105 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 04:49:27.248319 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":149,"threads_started":1}
I20260808 04:49:27.250414 16895 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41025
I20260808 04:49:27.250633 16895 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:27.251161 16895 heartbeater.cc:511] Master 127.22.2.190:41025 requested a full tablet report, sending...
I20260808 04:49:27.252162 16206 ts_manager.cc:194] Registered new tserver with Master: f4bf2ef272784ce5bbf8b28a14ce2504 (127.22.2.131:34617)
I20260808 04:49:27.252818 16206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.131:41065
I20260808 04:49:27.262979 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:49:27.268808 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.265103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.22.2.190:41025)
0808 04:49:27.265251 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.265256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.265458 (+ 202us) server_negotiation.cc:407] Connection header received
0808 04:49:27.265548 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.265550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.265587 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.265650 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:27.265877 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.266367 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.267131 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.267367 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.267446 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.268206 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:27.268209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:27.268211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:27.268223 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:27.268240 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:27.268248 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:27.268307 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.268506 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.268631 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 04:49:27.271989 16206 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:49:27.275249 16206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:49:27.290539 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.284088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:34617 (local address 127.0.0.1:57912)
0808 04:49:27.285520 (+ 1432us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:27.285533 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:27.285682 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:27.286180 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:27.286186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:27.286210 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:27.286319 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.286329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.287796 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.287799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.289050 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.289058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.289985 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.289989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.290095 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.290133 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 04:49:27.290258 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 04:49:27.290379 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1374,"mutex_wait_us":20,"thread_start_us":91,"threads_started":1}
I20260808 04:49:27.290546 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.284658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57912 (local address 127.22.2.131:34617)
0808 04:49:27.285079 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.285091 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.285677 (+ 586us) server_negotiation.cc:407] Connection header received
0808 04:49:27.285806 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.285812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.286013 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.286182 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:27.286446 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.287663 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.289178 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.289846 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.289982 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.290175 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.290287 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.290381 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":132,"threads_started":1}
I20260808 04:49:27.290951 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.284998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.22.2.129:38787)
0808 04:49:27.285412 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.285420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.286057 (+ 637us) server_negotiation.cc:407] Connection header received
0808 04:49:27.286161 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.286167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.286358 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.286531 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:27.286738 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.287818 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.289101 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.289873 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.289986 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.290174 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.290705 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.290805 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":88,"threads_started":1}
I20260808 04:49:27.291224 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.284593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.22.2.130:39201)
0808 04:49:27.285279 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.285294 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.285320 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:49:27.285583 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.285587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.285751 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.285918 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:27.286271 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.287209 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.289058 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.289873 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.289986 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.290174 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.290950 (+ 776us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.291060 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":166,"threads_started":1}
I20260808 04:49:27.291287 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.284830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:38787 (local address 127.0.0.1:40918)
0808 04:49:27.285859 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:27.285871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:27.285960 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:27.286497 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:27.286499 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:27.286517 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:27.286606 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.286612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.287958 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.287963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.288938 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.288950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.290246 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.290250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.290466 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.290534 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 04:49:27.290604 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 04:49:27.291149 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":555,"spinlock_wait_cycles":10368,"thread_start_us":97,"threads_started":1}
I20260808 04:49:27.290848 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.284049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:39201 (local address 127.0.0.1:51844)
0808 04:49:27.285208 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:27.285257 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:27.285401 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:27.285907 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:27.285913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:27.285950 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:27.286095 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.286109 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.287330 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.287333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.288918 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.288930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.290246 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.290250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.290461 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.290531 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 04:49:27.290604 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:49:27.290709 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"mutex_wait_us":99,"spinlock_wait_cycles":3456,"thread_start_us":66,"threads_started":1}
I20260808 04:49:27.294190 16795 tablet_service.cc:1467] Processing CreateTablet for tablet 4a874ac8f48140acb4d90c9d36c40a81 (DEFAULT_TABLE table=test-table [id=a431d809239a4ff9b6adc37867b354ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:27.294188 16390 tablet_service.cc:1467] Processing CreateTablet for tablet 4a874ac8f48140acb4d90c9d36c40a81 (DEFAULT_TABLE table=test-table [id=a431d809239a4ff9b6adc37867b354ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:27.295361 16795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a874ac8f48140acb4d90c9d36c40a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:27.295361 16390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a874ac8f48140acb4d90c9d36c40a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:27.297262 16592 tablet_service.cc:1467] Processing CreateTablet for tablet 4a874ac8f48140acb4d90c9d36c40a81 (DEFAULT_TABLE table=test-table [id=a431d809239a4ff9b6adc37867b354ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:27.298148 16592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a874ac8f48140acb4d90c9d36c40a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:27.302904 16918 tablet_bootstrap.cc:492] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c: Bootstrap starting.
I20260808 04:49:27.304226 16920 tablet_bootstrap.cc:492] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Bootstrap starting.
I20260808 04:49:27.304812 16921 tablet_bootstrap.cc:492] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Bootstrap starting.
I20260808 04:49:27.306113 16918 tablet_bootstrap.cc:654] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:27.306495 16920 tablet_bootstrap.cc:654] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:27.306911 16918 log.cc:824] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:27.307248 16921 tablet_bootstrap.cc:654] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:27.307322 16920 log.cc:824] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:27.308068 16921 log.cc:824] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:27.309036 16918 tablet_bootstrap.cc:492] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c: No bootstrap required, opened a new log
I20260808 04:49:27.309247 16918 ts_tablet_manager.cc:1397] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 04:49:27.309345 16920 tablet_bootstrap.cc:492] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: No bootstrap required, opened a new log
I20260808 04:49:27.309582 16920 ts_tablet_manager.cc:1397] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:49:27.310106 16921 tablet_bootstrap.cc:492] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: No bootstrap required, opened a new log
I20260808 04:49:27.310297 16921 ts_tablet_manager.cc:1397] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:49:27.313045 16918 raft_consensus.cc:348] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.313288 16918 raft_consensus.cc:374] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:27.313364 16918 raft_consensus.cc:729] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f999c6fe040463480842a845f5c849c, State: Initialized, Role: FOLLOWER
I20260808 04:49:27.313843 16918 consensus_queue.cc:260] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.313879 16920 raft_consensus.cc:348] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.314132 16920 raft_consensus.cc:374] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:27.314193 16920 raft_consensus.cc:729] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf2ef272784ce5bbf8b28a14ce2504, State: Initialized, Role: FOLLOWER
I20260808 04:49:27.314419 16921 raft_consensus.cc:348] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.314656 16921 raft_consensus.cc:374] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:27.314640 16920 consensus_queue.cc:260] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.314750 16921 raft_consensus.cc:729] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629a0ce3882c4df388cc6dd7f0deff2a, State: Initialized, Role: FOLLOWER
I20260808 04:49:27.314909 16918 ts_tablet_manager.cc:1428] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:49:27.315212 16921 consensus_queue.cc:260] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.315311 16491 heartbeater.cc:503] Master 127.22.2.190:41025 was elected leader, sending a full tablet report...
I20260808 04:49:27.315675 16920 ts_tablet_manager.cc:1428] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:49:27.315735 16895 heartbeater.cc:503] Master 127.22.2.190:41025 was elected leader, sending a full tablet report...
I20260808 04:49:27.316114 16921 ts_tablet_manager.cc:1428] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 04:49:27.316425 16685 heartbeater.cc:503] Master 127.22.2.190:41025 was elected leader, sending a full tablet report...
W20260808 04:49:27.490063 16897 tablet.cc:2369] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:49:27.503463 16687 tablet.cc:2369] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:27.517699 16927 raft_consensus.cc:482] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:27.517941 16927 raft_consensus.cc:504] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.519189 16927 leader_election.cc:288] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 629a0ce3882c4df388cc6dd7f0deff2a (127.22.2.130:39201), 2f999c6fe040463480842a845f5c849c (127.22.2.129:38787)
W20260808 04:49:27.521071 16494 tablet.cc:2369] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:27.523236 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.519374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:39201 (local address 127.22.2.131:37903)
0808 04:49:27.519515 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:27.519531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:27.519627 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:27.519941 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:27.519945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:27.519973 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:27.520113 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.520123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.521248 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.521251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.522141 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.522151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.522798 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.522802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.522906 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.522931 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 04:49:27.523023 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 04:49:27.523098 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":27}
I20260808 04:49:27.523236 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.519517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:37903 (local address 127.22.2.130:39201)
0808 04:49:27.519658 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.519663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.519694 (+ 31us) server_negotiation.cc:407] Connection header received
0808 04:49:27.519755 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.519761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.519814 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.519909 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:27.520240 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.521108 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.522277 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.522671 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.522804 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.522856 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.523042 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.523111 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 04:49:27.523469 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.519700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:41073 (local address 127.22.2.129:38787)
0808 04:49:27.519829 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.519835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.520036 (+ 201us) server_negotiation.cc:407] Connection header received
0808 04:49:27.520102 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.520104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.520145 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.520238 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:27.520509 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.521254 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.522449 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.522864 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.522999 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.523047 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.523227 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.523327 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 04:49:27.523693 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.519537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:38787 (local address 127.22.2.131:41073)
0808 04:49:27.519919 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:27.519932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:27.520031 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:27.520268 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:27.520275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:27.520293 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:27.520393 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.520398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.521372 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.521375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.522318 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:27.522329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.522991 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.522995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:27.523098 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.523117 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 04:49:27.523208 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 04:49:27.523278 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":22,"spinlock_wait_cycles":1408,"thread_start_us":119,"threads_started":1}
I20260808 04:49:27.524080 16617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a874ac8f48140acb4d90c9d36c40a81" candidate_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" is_pre_election: true
I20260808 04:49:27.524379 16424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a874ac8f48140acb4d90c9d36c40a81" candidate_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f999c6fe040463480842a845f5c849c" is_pre_election: true
I20260808 04:49:27.524447 16617 raft_consensus.cc:2454] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4bf2ef272784ce5bbf8b28a14ce2504 in term 0.
I20260808 04:49:27.524698 16424 raft_consensus.cc:2454] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4bf2ef272784ce5bbf8b28a14ce2504 in term 0.
I20260808 04:49:27.525075 16719 leader_election.cc:302] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629a0ce3882c4df388cc6dd7f0deff2a, f4bf2ef272784ce5bbf8b28a14ce2504; no voters:
I20260808 04:49:27.525377 16927 raft_consensus.cc:2781] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:49:27.525481 16927 raft_consensus.cc:482] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:49:27.525540 16927 raft_consensus.cc:3037] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:27.526618 16927 raft_consensus.cc:504] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.527014 16927 leader_election.cc:288] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [CANDIDATE]: Term 1 election: Requested vote from peers 629a0ce3882c4df388cc6dd7f0deff2a (127.22.2.130:39201), 2f999c6fe040463480842a845f5c849c (127.22.2.129:38787)
I20260808 04:49:27.527381 16617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a874ac8f48140acb4d90c9d36c40a81" candidate_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629a0ce3882c4df388cc6dd7f0deff2a"
I20260808 04:49:27.527382 16424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a874ac8f48140acb4d90c9d36c40a81" candidate_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f999c6fe040463480842a845f5c849c"
I20260808 04:49:27.527515 16424 raft_consensus.cc:3037] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:27.527515 16617 raft_consensus.cc:3037] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:27.528859 16617 raft_consensus.cc:2454] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4bf2ef272784ce5bbf8b28a14ce2504 in term 1.
I20260808 04:49:27.528859 16424 raft_consensus.cc:2454] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4bf2ef272784ce5bbf8b28a14ce2504 in term 1.
I20260808 04:49:27.529237 16720 leader_election.cc:302] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f999c6fe040463480842a845f5c849c, f4bf2ef272784ce5bbf8b28a14ce2504; no voters:
I20260808 04:49:27.529435 16927 raft_consensus.cc:2781] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:27.529666 16927 raft_consensus.cc:686] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 1 LEADER]: Becoming Leader. State: Replica: f4bf2ef272784ce5bbf8b28a14ce2504, State: Running, Role: LEADER
I20260808 04:49:27.530012 16927 consensus_queue.cc:237] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } }
I20260808 04:49:27.533600 16207 catalog_manager.cc:5495] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4bf2ef272784ce5bbf8b28a14ce2504 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4bf2ef272784ce5bbf8b28a14ce2504" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 } health_report { overall_health: UNKNOWN } } }
I20260808 04:49:27.580637 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.576980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57920 (local address 127.22.2.131:34617)
0808 04:49:27.577151 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:27.577156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:27.577171 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:49:27.577304 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:27.577307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:27.577365 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:27.577465 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:49:27.577677 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.578213 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.579074 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:27.579334 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:27.579424 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:27.579690 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:49:27.580155 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:49:27.580259 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:27.580389 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 04:49:27.580436 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 04:49:27.589298 16424 raft_consensus.cc:1261] T 4a874ac8f48140acb4d90c9d36c40a81 P 2f999c6fe040463480842a845f5c849c [term 1 FOLLOWER]: Refusing update from remote peer f4bf2ef272784ce5bbf8b28a14ce2504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:27.590044 16617 raft_consensus.cc:1261] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 1 FOLLOWER]: Refusing update from remote peer f4bf2ef272784ce5bbf8b28a14ce2504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:27.590094 16927 consensus_queue.cc:1035] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f999c6fe040463480842a845f5c849c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:27.591888 16927 consensus_queue.cc:1035] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629a0ce3882c4df388cc6dd7f0deff2a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 39201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:27.603487 16941 mvcc.cc:205] Tried to move back new op lower bound from 7316130068835274752 to 7316130068605751296. Current Snapshot: MvccSnapshot[applied={T|T < 7316130068835274752}]
I20260808 04:49:27.604592 16935 mvcc.cc:205] Tried to move back new op lower bound from 7316130068835274752 to 7316130068605751296. Current Snapshot: MvccSnapshot[applied={T|T < 7316130068835274752}]
I20260808 04:49:27.604683 16938 mvcc.cc:205] Tried to move back new op lower bound from 7316130068835274752 to 7316130068605751296. Current Snapshot: MvccSnapshot[applied={T|T < 7316130068835274752}]
I20260808 04:49:27.615394 16207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:27.615576 16207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:27.618546 16207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcfa74945a6f428b9482c4c4c9090095: Sending DeleteTablet for 3 replicas of tablet 4a874ac8f48140acb4d90c9d36c40a81
I20260808 04:49:27.619352 16592 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a874ac8f48140acb4d90c9d36c40a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:27 UTC) from {username='slave'} at 127.0.0.1:51844
I20260808 04:49:27.619377 16795 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a874ac8f48140acb4d90c9d36c40a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:27 UTC) from {username='slave'} at 127.0.0.1:57912
I20260808 04:49:27.620402 16952 tablet_replica.cc:330] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: stopping tablet replica
I20260808 04:49:27.620935 16390 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a874ac8f48140acb4d90c9d36c40a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:27 UTC) from {username='slave'} at 127.0.0.1:40918
I20260808 04:49:27.620998 16952 raft_consensus.cc:2229] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:27.621846 16952 raft_consensus.cc:2258] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:27.621887 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 16297
I20260808 04:49:27.632083 16952 ts_tablet_manager.cc:1905] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:27.634559 16952 ts_tablet_manager.cc:1918] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:27.634684 16952 log.cc:1196] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/4a874ac8f48140acb4d90c9d36c40a81
I20260808 04:49:27.635040 16952 ts_tablet_manager.cc:1939] T 4a874ac8f48140acb4d90c9d36c40a81 P 629a0ce3882c4df388cc6dd7f0deff2a: Deleting consensus metadata
I20260808 04:49:27.636279 16184 catalog_manager.cc:4841] TS 629a0ce3882c4df388cc6dd7f0deff2a (127.22.2.130:39201): tablet 4a874ac8f48140acb4d90c9d36c40a81 (table test-table [id=a431d809239a4ff9b6adc37867b354ee]) successfully deleted
I20260808 04:49:27.647231 16951 tablet_replica.cc:330] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: stopping tablet replica
I20260808 04:49:27.647627 16951 raft_consensus.cc:2229] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:49:27.648118 16951 raft_consensus.cc:2258] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 04:49:27.648504 16185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:38787 (error 108)
I20260808 04:49:27.649654 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 16497
I20260808 04:49:27.652839 16951 ts_tablet_manager.cc:1905] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:27.655457 16951 ts_tablet_manager.cc:1918] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:27.655557 16951 log.cc:1196] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/4a874ac8f48140acb4d90c9d36c40a81
I20260808 04:49:27.655879 16951 ts_tablet_manager.cc:1939] T 4a874ac8f48140acb4d90c9d36c40a81 P f4bf2ef272784ce5bbf8b28a14ce2504: Deleting consensus metadata
I20260808 04:49:27.658080 16183 catalog_manager.cc:4841] TS f4bf2ef272784ce5bbf8b28a14ce2504 (127.22.2.131:34617): tablet 4a874ac8f48140acb4d90c9d36c40a81 (table test-table [id=a431d809239a4ff9b6adc37867b354ee]) successfully deleted
I20260808 04:49:27.658192 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:27.657895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:38787 (local address 127.0.0.1:40922)
0808 04:49:27.658018 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:27.658105 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 04:49:27.658370 16185 catalog_manager.cc:4584] TS 2f999c6fe040463480842a845f5c849c (127.22.2.129:38787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a874ac8f48140acb4d90c9d36c40a81: Network error: Client connection negotiation failed: client connection to 127.22.2.129:38787: connect: Connection refused (error 111)
I20260808 04:49:27.663990 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 16689
I20260808 04:49:27.676071 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 16160
2026-08-08T04:49:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1397 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:27 dist-test-slave-04bb krb5kdc[16961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:27 dist-test-slave-04bb krb5kdc[16961](info): set up 2 sockets
Aug 08 04:49:27 dist-test-slave-04bb krb5kdc[16961](info): commencing operation
krb5kdc: starting...
W20260808 04:49:28.976866 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:29 dist-test-slave-04bb krb5kdc[16961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:49:29.103688 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35955
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:29.216689 16979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:29.216940 16979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:29.216980 16979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:29.220935 16979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:29.220983 16979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:29.221001 16979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:29.221050 16979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:29.221079 16979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:29.226177 16979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:29.227207 16979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:29.228263 16979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:29.236397 16986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:29.236409 16987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:29.236454 16990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:29.236692 16979 server_base.cc:1034] running on GCE node
I20260808 04:49:29.237289 16979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:29.238301 16979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:29.239470 16979 hybrid_clock.cc:648] HybridClock initialized: now 1786164569239451 us; error 31 us; skew 500 ppm
Aug 08 04:49:29 dist-test-slave-04bb krb5kdc[16961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:29.246234 16979 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:49:29.247079 16979 webserver.cc:460] Webserver started at http://127.22.2.190:45915/ using document root <none> and password file <none>
I20260808 04:49:29.247634 16979 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:29.247725 16979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:29.247972 16979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:29.250087 16979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "064f9ea482754441b4f1a01278056c44"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "29ad03575d6ac1d7dec3acffa9449b19"
server_key_iv: "3f5c4788b110732c450611eed26f16de"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.250646 16979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "064f9ea482754441b4f1a01278056c44"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "29ad03575d6ac1d7dec3acffa9449b19"
server_key_iv: "3f5c4788b110732c450611eed26f16de"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.254431 16979 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:49:29.256868 16997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:29.258219 16979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:49:29.258373 16979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "064f9ea482754441b4f1a01278056c44"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "29ad03575d6ac1d7dec3acffa9449b19"
server_key_iv: "3f5c4788b110732c450611eed26f16de"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.258493 16979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:29.282958 16979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:29.286128 16979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:29.286362 16979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:29.294284 16979 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35955
I20260808 04:49:29.294312 17079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35955 every 8 connection(s)
I20260808 04:49:29.295256 16979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:29.298527 17080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:29.299983 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 16979
I20260808 04:49:29.300149 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:29.300442 22538 external_mini_cluster.cc:1428] Setting key 0387297d7740ebfdf4e986d5836eb133
I20260808 04:49:29.304708 17080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44: Bootstrap starting.
I20260808 04:49:29.306921 17080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:29.307756 17080 log.cc:824] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:49:29 dist-test-slave-04bb krb5kdc[16961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:29.310055 17080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44: No bootstrap required, opened a new log
I20260808 04:49:29.313820 17080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064f9ea482754441b4f1a01278056c44" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35955 } }
I20260808 04:49:29.314154 17080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:29.314239 17080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064f9ea482754441b4f1a01278056c44, State: Initialized, Role: FOLLOWER
I20260808 04:49:29.314769 17080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064f9ea482754441b4f1a01278056c44" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35955 } }
I20260808 04:49:29.314929 17080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:29.314962 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:29.301946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.22.2.190:35955)
0808 04:49:29.302420 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:29.302436 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:29.302471 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:49:29.303083 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:29.303100 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:29.303411 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:29.303749 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:29.304578 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.305597 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.306471 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.306936 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.307121 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:29.310187 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:29.310196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:29.310224 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:29.310252 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:29.311926 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:29.312416 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:29.312421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:29.312425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:29.312582 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:29.312868 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:29.312872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:29.312873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:29.313295 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:29.313521 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:29.313780 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 04:49:29.314064 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":149,"threads_started":1}
I20260808 04:49:29.315024 17080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:29.315138 17080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:29.316241 17080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064f9ea482754441b4f1a01278056c44" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35955 } }
I20260808 04:49:29.316668 17080 leader_election.cc:302] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 064f9ea482754441b4f1a01278056c44; no voters:
I20260808 04:49:29.317019 17080 leader_election.cc:288] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:29.317106 17085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:29.317313 17085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [term 1 LEADER]: Becoming Leader. State: Replica: 064f9ea482754441b4f1a01278056c44, State: Running, Role: LEADER
I20260808 04:49:29.317605 17085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064f9ea482754441b4f1a01278056c44" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35955 } }
I20260808 04:49:29.318320 17080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:29.318822 17085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "064f9ea482754441b4f1a01278056c44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064f9ea482754441b4f1a01278056c44" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35955 } } }
I20260808 04:49:29.318930 17085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:29.319151 17085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 064f9ea482754441b4f1a01278056c44. Latest consensus state: current_term: 1 leader_uuid: "064f9ea482754441b4f1a01278056c44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064f9ea482754441b4f1a01278056c44" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35955 } } }
I20260808 04:49:29.319253 17085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:29.319691 17093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:29.322363 17093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:29.327740 17093 catalog_manager.cc:1347] Generated new cluster ID: 10b095d56ae04a718c67a5f9746e901a
I20260808 04:49:29.327804 17093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:29.337327 17093 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:29.338236 17093 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:29.347344 17093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44: Generated new TSK 0
I20260808 04:49:29.348016 17093 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:49:29.383574 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35955
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:29.496803 17114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:29.497027 17114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:29.497073 17114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:29.500922 17114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:29.500965 17114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:29.501019 17114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:49:29.505961 17114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:29.506848 17114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:29.507771 17114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:29.517222 17120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:29.517282 17123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:29.517338 17114 server_base.cc:1034] running on GCE node
W20260808 04:49:29.517186 17121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:29.517823 17114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:29.518493 17114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:29.519671 17114 hybrid_clock.cc:648] HybridClock initialized: now 1786164569519647 us; error 42 us; skew 500 ppm
Aug 08 04:49:29 dist-test-slave-04bb krb5kdc[16961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:29.522564 17114 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:49:29.523382 17114 webserver.cc:460] Webserver started at http://127.22.2.129:38051/ using document root <none> and password file <none>
I20260808 04:49:29.523904 17114 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:29.523964 17114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:29.524130 17114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:29.526219 17114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "91f5900979b34134a7a8f201017f8585"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "3857879a32a60763c9b7f48aeeec468d"
server_key_iv: "79bba932dcad963c1f00caf796272ee2"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.526744 17114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "91f5900979b34134a7a8f201017f8585"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "3857879a32a60763c9b7f48aeeec468d"
server_key_iv: "79bba932dcad963c1f00caf796272ee2"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.530597 17114 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 04:49:29.533047 17134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:29.534286 17114 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 04:49:29.534409 17114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "91f5900979b34134a7a8f201017f8585"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "3857879a32a60763c9b7f48aeeec468d"
server_key_iv: "79bba932dcad963c1f00caf796272ee2"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.534514 17114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:29.553332 17114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:29.556774 17114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:29.556931 17114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:29.557451 17114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:29.558259 17114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:29.558339 17114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:29.558395 17114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:29.558425 17114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:29.567737 17114 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:43621
I20260808 04:49:29.567813 17307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:43621 every 8 connection(s)
I20260808 04:49:29.568743 17114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:49:29.569772 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 17114
I20260808 04:49:29.569902 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:49:29.570276 22538 external_mini_cluster.cc:1428] Setting key 127dadb0188c2d49e39ddea0c4c66ca7
Aug 08 04:49:29 dist-test-slave-04bb krb5kdc[16961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 04:49:29.583278 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:29.570739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:43955 (local address 127.22.2.190:35955)
0808 04:49:29.570910 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:29.570915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:29.571768 (+ 853us) server_negotiation.cc:407] Connection header received
0808 04:49:29.573186 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:29.573190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:29.573263 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:29.573384 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:29.574544 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.575094 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.576260 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.576636 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.576769 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:29.579353 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:29.579357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:29.579360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:29.579386 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:29.580984 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:29.581532 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:29.581537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:29.581539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:29.581631 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:29.582004 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:29.582009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:29.582011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:29.582239 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:29.582399 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:29.582975 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 04:49:29.583102 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 04:49:29.585568 17315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:29.571164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35955 (local address 127.22.2.129:43955)
0808 04:49:29.571619 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:29.571673 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:29.572979 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:29.573564 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:29.573571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:29.573921 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:29.574395 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:29.574408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.575242 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.575245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:29.576109 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:29.576119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.576822 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.576827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:29.577006 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:29.577646 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:29.577662 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:29.579178 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:29.581124 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:29.581132 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:29.581149 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:29.581421 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:29.581750 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:29.581753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:29.581755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:29.581900 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:29.582399 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:29.582406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:29.582658 (+ 252us) client_negotiation.cc:769] Sending connection context
0808 04:49:29.582865 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 04:49:29.583195 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":135,"threads_started":1}
I20260808 04:49:29.587080 17308 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35955
I20260808 04:49:29.587373 17308 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:29.588021 17308 heartbeater.cc:511] Master 127.22.2.190:35955 requested a full tablet report, sending...
I20260808 04:49:29.589812 17022 ts_manager.cc:194] Registered new tserver with Master: 91f5900979b34134a7a8f201017f8585 (127.22.2.129:43621)
I20260808 04:49:29.590984 17022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:43955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:49:29.600629 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35955
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:29.712395 17319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:29.712651 17319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:29.712693 17319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:29.716550 17319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:29.716596 17319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:29.716698 17319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:49:29.722083 17319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:29.723038 17319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:29.724072 17319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:29.733835 17325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:29.733912 17326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:29.733856 17328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:29.734059 17319 server_base.cc:1034] running on GCE node
I20260808 04:49:29.734606 17319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:29.735275 17319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:29.736469 17319 hybrid_clock.cc:648] HybridClock initialized: now 1786164569736454 us; error 25 us; skew 500 ppm
Aug 08 04:49:29 dist-test-slave-04bb krb5kdc[16961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:29.739431 17319 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:49:29.740236 17319 webserver.cc:460] Webserver started at http://127.22.2.130:35683/ using document root <none> and password file <none>
I20260808 04:49:29.740823 17319 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:29.740887 17319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:29.741055 17319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:29.743252 17319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "3a93b1a06b26482ab522d535bc70f210"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "fd55afb1374b11d2e8c0cfe82f181644"
server_key_iv: "fa23469282aeb37920d984ed73e26a15"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.743768 17319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "3a93b1a06b26482ab522d535bc70f210"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "fd55afb1374b11d2e8c0cfe82f181644"
server_key_iv: "fa23469282aeb37920d984ed73e26a15"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.747413 17319 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:49:29.749886 17338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:29.751230 17319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:29.751338 17319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "3a93b1a06b26482ab522d535bc70f210"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "fd55afb1374b11d2e8c0cfe82f181644"
server_key_iv: "fa23469282aeb37920d984ed73e26a15"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.751426 17319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:29.783633 17319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:29.787506 17319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:29.787690 17319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:29.788323 17319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:29.789371 17319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:29.789424 17319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:29.789469 17319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:29.789487 17319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:29.799491 17319 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:43095
I20260808 04:49:29.799515 17506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:43095 every 8 connection(s)
I20260808 04:49:29.800508 17319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:49:29.808403 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 17319
I20260808 04:49:29.808617 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:49:29.808993 22538 external_mini_cluster.cc:1428] Setting key d77f859b1d613bf8c2eae5c205323c6e
Aug 08 04:49:29 dist-test-slave-04bb krb5kdc[16961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:29.816179 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:29.802683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:59391 (local address 127.22.2.190:35955)
0808 04:49:29.802834 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:29.802838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:29.803874 (+ 1036us) server_negotiation.cc:407] Connection header received
0808 04:49:29.805322 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:29.805325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:29.805388 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:29.805501 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:29.806735 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.807270 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.808476 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.808864 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.808985 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:29.812154 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:29.812158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:29.812161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:29.812188 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:29.813765 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:29.814445 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:29.814448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:29.814450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:29.814532 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:29.814968 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:29.814971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:29.814972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:29.815209 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:29.815293 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:29.815936 (+ 643us) server_negotiation.cc:299] Negotiation successful
0808 04:49:29.816045 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 04:49:29.817602 17511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:29.803109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35955 (local address 127.22.2.130:59391)
0808 04:49:29.803696 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:29.803755 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:29.805086 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:29.805689 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:29.805696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:29.806071 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:29.806566 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:29.806581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.807404 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.807410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:29.808338 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:29.808348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:29.809029 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:29.809034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:29.809294 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:29.810084 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:29.810104 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:29.811984 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:29.813894 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:29.813900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:29.813915 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:29.814321 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:29.814638 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:29.814643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:29.814646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:29.814862 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:29.815327 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:29.815337 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:29.815659 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 04:49:29.815889 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 04:49:29.816270 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":134,"threads_started":1}
I20260808 04:49:29.818990 17508 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35955
I20260808 04:49:29.819320 17508 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:29.820029 17508 heartbeater.cc:511] Master 127.22.2.190:35955 requested a full tablet report, sending...
I20260808 04:49:29.821362 17022 ts_manager.cc:194] Registered new tserver with Master: 3a93b1a06b26482ab522d535bc70f210 (127.22.2.130:43095)
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:49:29.822052 17022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:59391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:49:29.841804 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35955
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:29.956900 17516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:29.957150 17516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:29.957197 17516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:29.961191 17516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:29.961241 17516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:29.961330 17516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:49:29.966581 17516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:29.967573 17516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:29.968647 17516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:29.978449 17525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:29.978504 17529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:29.978542 17516 server_base.cc:1034] running on GCE node
W20260808 04:49:29.978462 17526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:29.979271 17516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:29.979897 17516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:29.981093 17516 hybrid_clock.cc:648] HybridClock initialized: now 1786164569981061 us; error 43 us; skew 500 ppm
Aug 08 04:49:29 dist-test-slave-04bb krb5kdc[16961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:29.983846 17516 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:49:29.984795 17516 webserver.cc:460] Webserver started at http://127.22.2.131:41407/ using document root <none> and password file <none>
I20260808 04:49:29.985409 17516 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:29.985502 17516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:29.985745 17516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:29.987769 17516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "d5cea30ceecc4900be07c8575c11b36e"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "56eceeb7578fbb0e1802525a562b6855"
server_key_iv: "148456e0ce64a258fd19b9eea3f825e7"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.988365 17516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "d5cea30ceecc4900be07c8575c11b36e"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "56eceeb7578fbb0e1802525a562b6855"
server_key_iv: "148456e0ce64a258fd19b9eea3f825e7"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.992062 17516 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:49:29.994596 17537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:29.995877 17516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:29.996012 17516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "d5cea30ceecc4900be07c8575c11b36e"
format_stamp: "Formatted at 2026-08-08 04:49:29 on dist-test-slave-04bb"
server_key: "56eceeb7578fbb0e1802525a562b6855"
server_key_iv: "148456e0ce64a258fd19b9eea3f825e7"
server_key_version: "encryptionkey@0"
I20260808 04:49:29.996119 17516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:30.009070 17516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:30.012039 17516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:30.012239 17516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:30.012833 17516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:30.013655 17516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:30.013727 17516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:30.013799 17516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:30.013844 17516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:30.023166 17516 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:45521
I20260808 04:49:30.023187 17700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:45521 every 8 connection(s)
I20260808 04:49:30.024071 17516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:49:30.028756 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 17516
I20260808 04:49:30.028913 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:49:30.029210 22538 external_mini_cluster.cc:1428] Setting key 7cc6c49d7da59124322878707c01427f
Aug 08 04:49:30 dist-test-slave-04bb krb5kdc[16961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164569, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:30.038502 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.025970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:45385 (local address 127.22.2.190:35955)
0808 04:49:30.026140 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.026144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.027032 (+ 888us) server_negotiation.cc:407] Connection header received
0808 04:49:30.028430 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.028434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.028497 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.028665 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:30.030130 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.030790 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.031969 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.032209 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.032319 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.035021 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:30.035024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:30.035026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:30.035051 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:30.036396 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:30.036947 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:30.036952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:30.036953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:30.037038 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:30.037388 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:30.037392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:30.037394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:30.037578 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:30.037691 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.038197 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.038332 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 04:49:30.039408 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.026369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35955 (local address 127.22.2.131:45385)
0808 04:49:30.026871 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:30.026914 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:30.028203 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:30.028856 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:30.028885 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:30.029306 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:30.029994 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.030017 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.030924 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.030928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.031820 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.031830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.032370 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.032375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.032623 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.033246 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:30.033263 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:30.034852 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:30.036518 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:30.036543 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:30.036558 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:30.036837 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:30.037135 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:30.037139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:30.037141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:30.037287 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:30.037698 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:30.037707 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:30.037939 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 04:49:30.038132 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 04:49:30.038471 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":132,"threads_started":1}
I20260808 04:49:30.040570 17701 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35955
I20260808 04:49:30.040843 17701 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:30.041342 17701 heartbeater.cc:511] Master 127.22.2.190:35955 requested a full tablet report, sending...
I20260808 04:49:30.042379 17021 ts_manager.cc:194] Registered new tserver with Master: d5cea30ceecc4900be07c8575c11b36e (127.22.2.131:45521)
I20260808 04:49:30.042975 17021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:45385
I20260808 04:49:30.043210 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:49:30.052582 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.045410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41538 (local address 127.22.2.190:35955)
0808 04:49:30.045532 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.045536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.045738 (+ 202us) server_negotiation.cc:407] Connection header received
0808 04:49:30.045832 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.045835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.045870 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.045937 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:30.046636 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.047121 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.047918 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.048122 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.048201 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.049677 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:30.049680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:30.049682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:30.049704 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:30.050955 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:30.051450 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:30.051453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:30.051455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:30.051544 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:30.051844 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:30.051846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:30.051848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:30.052025 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:30.052140 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.052343 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.052416 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:49:30.055756 17021 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:49:30.058948 17021 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:49:30.073889 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.067718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43621 (local address 127.0.0.1:37474)
0808 04:49:30.068491 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:30.068560 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:30.068712 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:30.069558 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:30.069564 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:30.069600 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:30.069885 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.069892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.071027 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.071031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.072351 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.072361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.073309 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.073313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.073432 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.073471 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 04:49:30.073611 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 04:49:30.073733 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":129,"spinlock_wait_cycles":5632,"thread_start_us":76,"threads_started":1}
I20260808 04:49:30.073895 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.067881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.22.2.129:43621)
0808 04:49:30.068302 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.068320 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.068879 (+ 559us) server_negotiation.cc:407] Connection header received
0808 04:49:30.068968 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.068975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.069189 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.069351 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:30.070120 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.070887 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.072489 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.073177 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.073307 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.073467 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.073648 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.073752 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":83,"threads_started":1}
I20260808 04:49:30.074038 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.068033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.22.2.130:43095)
0808 04:49:30.068495 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.068511 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.068836 (+ 325us) server_negotiation.cc:407] Connection header received
0808 04:49:30.069081 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.069087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.069244 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.069371 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:30.070035 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.070790 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.072352 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.073256 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.073335 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.073463 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.073787 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.073894 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":95,"threads_started":1}
I20260808 04:49:30.074440 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.067727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:43095 (local address 127.0.0.1:50142)
0808 04:49:30.068732 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:30.068746 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:30.068915 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:30.069558 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:30.069564 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:30.069601 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:30.069884 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.069892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.070922 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.070929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.072208 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.072221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.073558 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.073561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.073644 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.073663 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 04:49:30.074230 (+ 567us) client_negotiation.cc:240] Negotiation successful
0808 04:49:30.074302 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":170,"spinlock_wait_cycles":7040,"thread_start_us":131,"threads_started":1}
I20260808 04:49:30.075346 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.068462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:45521 (local address 127.0.0.1:43776)
0808 04:49:30.069243 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:30.069255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:30.069342 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:30.070954 (+ 1612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:30.070956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:30.070985 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:30.071093 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.071099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.072745 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.072749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.073829 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.073838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.074934 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.074939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.075017 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.075045 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 04:49:30.075124 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 04:49:30.075212 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":5,"thread_start_us":149,"threads_started":1}
I20260808 04:49:30.075990 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.068676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.22.2.131:45521)
0808 04:49:30.069650 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.069660 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.069679 (+ 19us) server_negotiation.cc:407] Connection header received
0808 04:49:30.069741 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.069745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.069894 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.070080 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:30.071232 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.072592 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.073956 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.074810 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.075457 (+ 647us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.075781 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.075823 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.075886 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":153,"threads_started":1}
I20260808 04:49:30.077579 17210 tablet_service.cc:1467] Processing CreateTablet for tablet ebe36e4f299047e899751e5c3c69634c (DEFAULT_TABLE table=test-table [id=a88f910afb374c88b4ab2c3d65015d0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:30.077579 17409 tablet_service.cc:1467] Processing CreateTablet for tablet ebe36e4f299047e899751e5c3c69634c (DEFAULT_TABLE table=test-table [id=a88f910afb374c88b4ab2c3d65015d0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:30.078229 17602 tablet_service.cc:1467] Processing CreateTablet for tablet ebe36e4f299047e899751e5c3c69634c (DEFAULT_TABLE table=test-table [id=a88f910afb374c88b4ab2c3d65015d0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:30.078733 17210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe36e4f299047e899751e5c3c69634c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:30.078733 17409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe36e4f299047e899751e5c3c69634c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:30.079135 17602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe36e4f299047e899751e5c3c69634c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:30.085711 17725 tablet_bootstrap.cc:492] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Bootstrap starting.
I20260808 04:49:30.086979 17729 tablet_bootstrap.cc:492] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Bootstrap starting.
I20260808 04:49:30.088124 17725 tablet_bootstrap.cc:654] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:30.088891 17726 tablet_bootstrap.cc:492] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585: Bootstrap starting.
I20260808 04:49:30.089310 17729 tablet_bootstrap.cc:654] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:30.089332 17725 log.cc:824] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:30.090178 17729 log.cc:824] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:30.091037 17726 tablet_bootstrap.cc:654] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:30.091414 17725 tablet_bootstrap.cc:492] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: No bootstrap required, opened a new log
I20260808 04:49:30.091848 17726 log.cc:824] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:30.092295 17729 tablet_bootstrap.cc:492] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: No bootstrap required, opened a new log
I20260808 04:49:30.092521 17729 ts_tablet_manager.cc:1397] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 04:49:30.093142 17725 ts_tablet_manager.cc:1397] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260808 04:49:30.096630 17729 raft_consensus.cc:348] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.096935 17729 raft_consensus.cc:374] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:30.096987 17729 raft_consensus.cc:729] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a93b1a06b26482ab522d535bc70f210, State: Initialized, Role: FOLLOWER
I20260808 04:49:30.097458 17729 consensus_queue.cc:260] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.098474 17729 ts_tablet_manager.cc:1428] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260808 04:49:30.098626 17508 heartbeater.cc:503] Master 127.22.2.190:35955 was elected leader, sending a full tablet report...
I20260808 04:49:30.098693 17725 raft_consensus.cc:348] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.098971 17725 raft_consensus.cc:374] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:30.099031 17725 raft_consensus.cc:729] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5cea30ceecc4900be07c8575c11b36e, State: Initialized, Role: FOLLOWER
I20260808 04:49:30.099406 17725 consensus_queue.cc:260] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.100108 17701 heartbeater.cc:503] Master 127.22.2.190:35955 was elected leader, sending a full tablet report...
I20260808 04:49:30.100381 17725 ts_tablet_manager.cc:1428] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 04:49:30.101151 17726 tablet_bootstrap.cc:492] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585: No bootstrap required, opened a new log
I20260808 04:49:30.101394 17726 ts_tablet_manager.cc:1397] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260808 04:49:30.105221 17726 raft_consensus.cc:348] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.105406 17726 raft_consensus.cc:374] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:30.105487 17726 raft_consensus.cc:729] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f5900979b34134a7a8f201017f8585, State: Initialized, Role: FOLLOWER
I20260808 04:49:30.105857 17726 consensus_queue.cc:260] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.106526 17308 heartbeater.cc:503] Master 127.22.2.190:35955 was elected leader, sending a full tablet report...
I20260808 04:49:30.106698 17726 ts_tablet_manager.cc:1428] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 04:49:30.275151 17703 tablet.cc:2369] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:49:30.301759 17509 tablet.cc:2369] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:49:30.320348 17312 tablet.cc:2369] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:30.421073 17736 raft_consensus.cc:482] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:30.421332 17736 raft_consensus.cc:504] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.422582 17736 leader_election.cc:288] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a93b1a06b26482ab522d535bc70f210 (127.22.2.130:43095), d5cea30ceecc4900be07c8575c11b36e (127.22.2.131:45521)
I20260808 04:49:30.426966 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.422956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:40387 (local address 127.22.2.130:43095)
0808 04:49:30.423112 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.423116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.423320 (+ 204us) server_negotiation.cc:407] Connection header received
0808 04:49:30.423520 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.423524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.423592 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.423640 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:30.424059 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.424877 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.425971 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.426376 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.426520 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.426576 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.426786 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.426863 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:49:30.426999 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.422855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:43095 (local address 127.22.2.129:40387)
0808 04:49:30.423202 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:30.423220 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:30.423357 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:30.423773 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:30.423775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:30.423835 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:30.423944 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.423949 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.424989 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.424992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.425826 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.425835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.426521 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.426525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.426630 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.426650 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 04:49:30.426743 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 04:49:30.426821 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":1,"spinlock_wait_cycles":10752,"thread_start_us":137,"threads_started":1}
I20260808 04:49:30.427137 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.423502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:58621 (local address 127.22.2.131:45521)
0808 04:49:30.423629 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.423634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.423757 (+ 123us) server_negotiation.cc:407] Connection header received
0808 04:49:30.423968 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.423977 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.424050 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.424134 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:30.424400 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.425103 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.426149 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.426556 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.426686 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.426742 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.426933 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.427004 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 04:49:30.427162 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.423328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:45521 (local address 127.22.2.129:58621)
0808 04:49:30.423638 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:30.423654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:30.423847 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:30.424155 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:30.424157 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:30.424172 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:30.424285 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.424289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.425218 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.425222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.426014 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:30.426022 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.426681 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.426685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:30.426793 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.426816 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 04:49:30.426916 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 04:49:30.426986 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":1,"spinlock_wait_cycles":21504,"thread_start_us":86,"threads_started":1}
I20260808 04:49:30.427726 17631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe36e4f299047e899751e5c3c69634c" candidate_uuid: "91f5900979b34134a7a8f201017f8585" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5cea30ceecc4900be07c8575c11b36e" is_pre_election: true
I20260808 04:49:30.427752 17439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe36e4f299047e899751e5c3c69634c" candidate_uuid: "91f5900979b34134a7a8f201017f8585" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a93b1a06b26482ab522d535bc70f210" is_pre_election: true
I20260808 04:49:30.428067 17439 raft_consensus.cc:2454] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f5900979b34134a7a8f201017f8585 in term 0.
I20260808 04:49:30.428054 17631 raft_consensus.cc:2454] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f5900979b34134a7a8f201017f8585 in term 0.
I20260808 04:49:30.428611 17140 leader_election.cc:302] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f5900979b34134a7a8f201017f8585, d5cea30ceecc4900be07c8575c11b36e; no voters:
I20260808 04:49:30.428923 17736 raft_consensus.cc:2781] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:49:30.429003 17736 raft_consensus.cc:482] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:49:30.429033 17736 raft_consensus.cc:3037] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:30.430305 17736 raft_consensus.cc:504] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.430713 17736 leader_election.cc:288] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [CANDIDATE]: Term 1 election: Requested vote from peers 3a93b1a06b26482ab522d535bc70f210 (127.22.2.130:43095), d5cea30ceecc4900be07c8575c11b36e (127.22.2.131:45521)
I20260808 04:49:30.431147 17631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe36e4f299047e899751e5c3c69634c" candidate_uuid: "91f5900979b34134a7a8f201017f8585" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5cea30ceecc4900be07c8575c11b36e"
I20260808 04:49:30.431175 17439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe36e4f299047e899751e5c3c69634c" candidate_uuid: "91f5900979b34134a7a8f201017f8585" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a93b1a06b26482ab522d535bc70f210"
I20260808 04:49:30.431270 17439 raft_consensus.cc:3037] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:30.431280 17631 raft_consensus.cc:3037] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:30.432303 17631 raft_consensus.cc:2454] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f5900979b34134a7a8f201017f8585 in term 1.
I20260808 04:49:30.432379 17439 raft_consensus.cc:2454] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f5900979b34134a7a8f201017f8585 in term 1.
I20260808 04:49:30.432624 17140 leader_election.cc:302] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f5900979b34134a7a8f201017f8585, d5cea30ceecc4900be07c8575c11b36e; no voters:
I20260808 04:49:30.432809 17736 raft_consensus.cc:2781] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:30.433048 17736 raft_consensus.cc:686] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [term 1 LEADER]: Becoming Leader. State: Replica: 91f5900979b34134a7a8f201017f8585, State: Running, Role: LEADER
I20260808 04:49:30.433398 17736 consensus_queue.cc:237] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } }
I20260808 04:49:30.436422 17021 catalog_manager.cc:5495] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f5900979b34134a7a8f201017f8585 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "91f5900979b34134a7a8f201017f8585" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f5900979b34134a7a8f201017f8585" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 } health_report { overall_health: UNKNOWN } } }
I20260808 04:49:30.528112 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.524178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.22.2.129:43621)
0808 04:49:30.524343 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.524348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.524365 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:49:30.524571 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.524574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.524637 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.524772 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:49:30.525043 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.525550 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.526570 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.526794 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.526908 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.527238 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:49:30.527642 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:49:30.527821 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.527924 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.527966 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 04:49:30.534327 17631 raft_consensus.cc:1261] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 1 FOLLOWER]: Refusing update from remote peer 91f5900979b34134a7a8f201017f8585: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:30.534372 17439 raft_consensus.cc:1261] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 1 FOLLOWER]: Refusing update from remote peer 91f5900979b34134a7a8f201017f8585: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:30.535054 17739 consensus_queue.cc:1035] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5cea30ceecc4900be07c8575c11b36e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:30.535364 17736 consensus_queue.cc:1035] T ebe36e4f299047e899751e5c3c69634c P 91f5900979b34134a7a8f201017f8585 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a93b1a06b26482ab522d535bc70f210" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 43095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:30.544104 17747 mvcc.cc:205] Tried to move back new op lower bound from 7316130080901165056 to 7316130080497631232. Current Snapshot: MvccSnapshot[applied={T|T < 7316130080901165056}]
I20260808 04:49:30.551755 17021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:30.551895 17021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:30.554338 17021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 064f9ea482754441b4f1a01278056c44: Sending DeleteTablet for 3 replicas of tablet ebe36e4f299047e899751e5c3c69634c
I20260808 04:49:30.555063 17210 tablet_service.cc:1514] Processing DeleteTablet for tablet ebe36e4f299047e899751e5c3c69634c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:30 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:37474
I20260808 04:49:30.555341 17409 tablet_service.cc:1514] Processing DeleteTablet for tablet ebe36e4f299047e899751e5c3c69634c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:30 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:50142
I20260808 04:49:30.555621 17602 tablet_service.cc:1514] Processing DeleteTablet for tablet ebe36e4f299047e899751e5c3c69634c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:30 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:43776
I20260808 04:49:30.555874 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 17114
I20260808 04:49:30.567142 17757 tablet_replica.cc:330] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: stopping tablet replica
I20260808 04:49:30.567207 17756 tablet_replica.cc:330] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: stopping tablet replica
I20260808 04:49:30.567539 17757 raft_consensus.cc:2229] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:30.567597 17756 raft_consensus.cc:2229] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:30.567919 17757 raft_consensus.cc:2258] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:30.568009 17756 raft_consensus.cc:2258] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:30.569651 17756 ts_tablet_manager.cc:1905] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:30.569651 17757 ts_tablet_manager.cc:1905] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:30.572444 17757 ts_tablet_manager.cc:1918] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:30.572444 17756 ts_tablet_manager.cc:1918] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:30.572572 17757 log.cc:1196] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/ebe36e4f299047e899751e5c3c69634c
I20260808 04:49:30.572573 17756 log.cc:1196] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/ebe36e4f299047e899751e5c3c69634c
I20260808 04:49:30.572914 17757 ts_tablet_manager.cc:1939] T ebe36e4f299047e899751e5c3c69634c P d5cea30ceecc4900be07c8575c11b36e: Deleting consensus metadata
I20260808 04:49:30.572914 17756 ts_tablet_manager.cc:1939] T ebe36e4f299047e899751e5c3c69634c P 3a93b1a06b26482ab522d535bc70f210: Deleting consensus metadata
I20260808 04:49:30.574211 17001 catalog_manager.cc:4841] TS d5cea30ceecc4900be07c8575c11b36e (127.22.2.131:45521): tablet ebe36e4f299047e899751e5c3c69634c (table test-table [id=a88f910afb374c88b4ab2c3d65015d0a]) successfully deleted
W20260808 04:49:30.574314 16998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:43621 (error 108)
I20260808 04:49:30.574586 17001 catalog_manager.cc:4841] TS 3a93b1a06b26482ab522d535bc70f210 (127.22.2.130:43095): tablet ebe36e4f299047e899751e5c3c69634c (table test-table [id=a88f910afb374c88b4ab2c3d65015d0a]) successfully deleted
I20260808 04:49:30.575315 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 17319
I20260808 04:49:30.575654 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.575289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43621 (local address 127.0.0.1:37498)
0808 04:49:30.575446 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:30.575532 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:43621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 04:49:30.579798 16998 catalog_manager.cc:4584] TS 91f5900979b34134a7a8f201017f8585 (127.22.2.129:43621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ebe36e4f299047e899751e5c3c69634c: Network error: Client connection negotiation failed: client connection to 127.22.2.129:43621: connect: Connection refused (error 111)
I20260808 04:49:30.588429 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 17516
I20260808 04:49:30.599602 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 16979
2026-08-08T04:49:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2924 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T04:49:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:30Z Disabled control of system clock
I20260808 04:49:30.637959 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35363
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:40715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 04:49:30.749948 17764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:30.750196 17764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:30.750229 17764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:30.754091 17764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:30.754137 17764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:30.754153 17764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:30.754168 17764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:30.754184 17764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:30.758859 17764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:30.759786 17764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:30.761113 17764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:30.769312 17771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:30.769320 17772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:30.769650 17764 server_base.cc:1034] running on GCE node
W20260808 04:49:30.769356 17774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:30.770499 17764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:30.771588 17764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:30.772773 17764 hybrid_clock.cc:648] HybridClock initialized: now 1786164570772749 us; error 47 us; skew 500 ppm
I20260808 04:49:30.774693 17764 webserver.cc:460] Webserver started at http://127.22.2.190:46129/ using document root <none> and password file <none>
I20260808 04:49:30.775329 17764 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:30.775429 17764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:30.775666 17764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:30.777746 17764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "5c35d25257b444a7964bac151d325d34"
format_stamp: "Formatted at 2026-08-08 04:49:30 on dist-test-slave-04bb"
server_key: "ef5c673e44fecb5ac76d7d0876cfc087"
server_key_iv: "20e783b3a148bffe00491347e9845a9d"
server_key_version: "encryptionkey@0"
I20260808 04:49:30.778316 17764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "5c35d25257b444a7964bac151d325d34"
format_stamp: "Formatted at 2026-08-08 04:49:30 on dist-test-slave-04bb"
server_key: "ef5c673e44fecb5ac76d7d0876cfc087"
server_key_iv: "20e783b3a148bffe00491347e9845a9d"
server_key_version: "encryptionkey@0"
I20260808 04:49:30.782279 17764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:49:30.784904 17785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:30.786021 17764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:49:30.786163 17764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "5c35d25257b444a7964bac151d325d34"
format_stamp: "Formatted at 2026-08-08 04:49:30 on dist-test-slave-04bb"
server_key: "ef5c673e44fecb5ac76d7d0876cfc087"
server_key_iv: "20e783b3a148bffe00491347e9845a9d"
server_key_version: "encryptionkey@0"
I20260808 04:49:30.786290 17764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:30.819772 17764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:30.820628 17764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:30.820837 17764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:30.828239 17764 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35363
I20260808 04:49:30.828243 17856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35363 every 8 connection(s)
I20260808 04:49:30.829319 17764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:30.832625 17857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:30.833833 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 17764
I20260808 04:49:30.833986 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:30.834307 22538 external_mini_cluster.cc:1428] Setting key c5764d146ed4e170ed4757225ce5eaad
I20260808 04:49:30.838821 17857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34: Bootstrap starting.
I20260808 04:49:30.841087 17857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:30.841913 17857 log.cc:824] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:30.844266 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:30.835762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.22.2.190:35363)
0808 04:49:30.836445 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:30.836461 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:30.836498 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:49:30.837224 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:30.837242 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:30.837651 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:30.838001 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:30.838438 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.839445 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.840337 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:30.841085 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:30.841790 (+ 705us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:30.842332 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:30.842340 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:30.842359 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:30.842875 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:30.842991 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:30.843021 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:30.843123 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:30.843293 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 04:49:30.843544 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":142,"threads_started":1}
I20260808 04:49:30.844403 17857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34: No bootstrap required, opened a new log
I20260808 04:49:30.848670 17857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c35d25257b444a7964bac151d325d34" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35363 } }
I20260808 04:49:30.848904 17857 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:30.848982 17857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c35d25257b444a7964bac151d325d34, State: Initialized, Role: FOLLOWER
I20260808 04:49:30.849390 17857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c35d25257b444a7964bac151d325d34" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35363 } }
I20260808 04:49:30.849539 17857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:30.849630 17857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:30.849731 17857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:30.850594 17857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c35d25257b444a7964bac151d325d34" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35363 } }
I20260808 04:49:30.850940 17857 leader_election.cc:302] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c35d25257b444a7964bac151d325d34; no voters:
I20260808 04:49:30.851253 17857 leader_election.cc:288] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:30.851415 17868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:30.851663 17868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [term 1 LEADER]: Becoming Leader. State: Replica: 5c35d25257b444a7964bac151d325d34, State: Running, Role: LEADER
I20260808 04:49:30.851994 17868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c35d25257b444a7964bac151d325d34" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35363 } }
I20260808 04:49:30.852571 17857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:30.853683 17869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c35d25257b444a7964bac151d325d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c35d25257b444a7964bac151d325d34" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35363 } } }
I20260808 04:49:30.853820 17869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:30.854130 17870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c35d25257b444a7964bac151d325d34. Latest consensus state: current_term: 1 leader_uuid: "5c35d25257b444a7964bac151d325d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c35d25257b444a7964bac151d325d34" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35363 } } }
I20260808 04:49:30.854219 17870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:30.854346 17884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:30.857479 17884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:30.862967 17884 catalog_manager.cc:1347] Generated new cluster ID: 3501a795c0be4e559a3c324ceeab31bb
I20260808 04:49:30.863029 17884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:30.877254 17884 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:30.878157 17884 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:30.887425 17884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34: Generated new TSK 0
I20260808 04:49:30.888193 17884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:30.893359 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35363
--builtin_ntp_servers=127.22.2.148:40715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 04:49:31.007166 17899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:31.007393 17899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:31.007429 17899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:31.011353 17899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:31.011401 17899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:31.011452 17899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:49:31.015933 17899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:31.016883 17899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:31.018150 17899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:31.028141 17905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:31.028151 17906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:31.028205 17909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:31.028707 17899 server_base.cc:1034] running on GCE node
I20260808 04:49:31.029173 17899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:31.029875 17899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:31.031059 17899 hybrid_clock.cc:648] HybridClock initialized: now 1786164571031037 us; error 35 us; skew 500 ppm
I20260808 04:49:31.033270 17899 webserver.cc:460] Webserver started at http://127.22.2.129:34261/ using document root <none> and password file <none>
I20260808 04:49:31.033914 17899 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:31.033984 17899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:31.034271 17899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:31.036463 17899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "fbe05bfe20f84d699d3c80ebf999f03e"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "f1d7d0410eac951f7c680fcc1dfc8507"
server_key_iv: "ee97689975a7d41d4c3cd6ddbbfed97b"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.037130 17899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "fbe05bfe20f84d699d3c80ebf999f03e"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "f1d7d0410eac951f7c680fcc1dfc8507"
server_key_iv: "ee97689975a7d41d4c3cd6ddbbfed97b"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.041085 17899 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:49:31.043576 17917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.044982 17899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:49:31.045133 17899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "fbe05bfe20f84d699d3c80ebf999f03e"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "f1d7d0410eac951f7c680fcc1dfc8507"
server_key_iv: "ee97689975a7d41d4c3cd6ddbbfed97b"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.045256 17899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:31.064041 17899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:31.064803 17899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:31.065001 17899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:31.065577 17899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:31.066540 17899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:31.066625 17899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.066701 17899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:31.066737 17899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.076516 17899 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33297
I20260808 04:49:31.076578 18068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33297 every 8 connection(s)
I20260808 04:49:31.077437 17899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:49:31.080848 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 17899
I20260808 04:49:31.080979 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:49:31.081352 22538 external_mini_cluster.cc:1428] Setting key dbfdfa6b2486bf35564225e637d6af2d
I20260808 04:49:31.083650 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35363
--builtin_ntp_servers=127.22.2.148:40715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 04:49:31.088048 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.079457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34775 (local address 127.22.2.190:35363)
0808 04:49:31.079619 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.079625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.080742 (+ 1117us) server_negotiation.cc:407] Connection header received
0808 04:49:31.082544 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.082548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.082599 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.082716 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:31.083971 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.084507 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.086027 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.086280 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.086396 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.087514 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:31.087518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:31.087520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:31.087534 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:31.087555 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:31.087572 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:31.087652 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.087836 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.087906 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 04:49:31.088932 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.079839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35363 (local address 127.22.2.129:34775)
0808 04:49:31.080498 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.080608 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:31.082310 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:31.082905 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:31.082912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:31.083045 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:31.083804 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.083830 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.084642 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.084649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.085894 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.085904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.086435 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.086439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.086603 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.087254 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:31.087256 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:31.087295 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:31.087357 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:31.087654 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:31.087673 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:31.087685 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 04:49:31.087808 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 04:49:31.088091 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":119,"threads_started":1}
I20260808 04:49:31.090195 18069 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35363
I20260808 04:49:31.090423 18069 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:31.091004 18069 heartbeater.cc:511] Master 127.22.2.190:35363 requested a full tablet report, sending...
I20260808 04:49:31.092712 17809 ts_manager.cc:194] Registered new tserver with Master: fbe05bfe20f84d699d3c80ebf999f03e (127.22.2.129:33297)
I20260808 04:49:31.094097 17809 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.129:34775
W20260808 04:49:31.197746 18073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:31.198014 18073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:31.198086 18073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:31.202009 18073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:31.202087 18073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:31.202176 18073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:49:31.206815 18073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:31.207845 18073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:31.209262 18073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:31.218868 18089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:31.218878 18083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:31.218878 18085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:31.219363 18073 server_base.cc:1034] running on GCE node
I20260808 04:49:31.219771 18073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:31.220468 18073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:31.221638 18073 hybrid_clock.cc:648] HybridClock initialized: now 1786164571221624 us; error 27 us; skew 500 ppm
I20260808 04:49:31.223604 18073 webserver.cc:460] Webserver started at http://127.22.2.130:35729/ using document root <none> and password file <none>
I20260808 04:49:31.224231 18073 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:31.224331 18073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:31.224627 18073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:31.226684 18073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "53c642ea1bde49d3b2b095df4b667c69"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "2719a0cc1dc3a0871ddd582c3a392ace"
server_key_iv: "013624dccd3a897fdadfaeb22e4d8b18"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.227275 18073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "53c642ea1bde49d3b2b095df4b667c69"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "2719a0cc1dc3a0871ddd582c3a392ace"
server_key_iv: "013624dccd3a897fdadfaeb22e4d8b18"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.231165 18073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 04:49:31.233855 18099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.235208 18073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:31.235356 18073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "53c642ea1bde49d3b2b095df4b667c69"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "2719a0cc1dc3a0871ddd582c3a392ace"
server_key_iv: "013624dccd3a897fdadfaeb22e4d8b18"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.235476 18073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:31.269440 18073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:31.270228 18073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:31.270455 18073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:31.271181 18073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:31.272236 18073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:31.272313 18073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.272406 18073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:31.272469 18073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.282088 18073 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:40759
I20260808 04:49:31.282236 18262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:40759 every 8 connection(s)
I20260808 04:49:31.283147 18073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:49:31.291307 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 18073
I20260808 04:49:31.291463 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:49:31.291850 22538 external_mini_cluster.cc:1428] Setting key 0d338ae637e98aad37f77206101300e4
I20260808 04:49:31.293581 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.285142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:51173 (local address 127.22.2.190:35363)
0808 04:49:31.285315 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.285321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.286449 (+ 1128us) server_negotiation.cc:407] Connection header received
0808 04:49:31.287901 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.287904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.287954 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.288033 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:31.289074 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.289672 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.291099 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.291536 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.291709 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.292988 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:31.292991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:31.292993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:31.293007 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:31.293027 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:31.293036 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:31.293122 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.293360 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.293429 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 04:49:31.294111 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35363
--builtin_ntp_servers=127.22.2.148:40715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 04:49:31.294565 18265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.285586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35363 (local address 127.22.2.130:51173)
0808 04:49:31.286292 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.286335 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:31.287672 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:31.288208 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:31.288215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:31.288329 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:31.288928 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.288943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.289818 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.289821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.290946 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.290957 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.291706 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.291709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.291864 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.292660 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:31.292662 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:31.292720 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:31.292815 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:31.293139 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:31.293151 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:31.293160 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 04:49:31.293361 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 04:49:31.293663 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":118,"threads_started":1}
I20260808 04:49:31.295981 18263 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35363
I20260808 04:49:31.296303 18263 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:31.296996 18263 heartbeater.cc:511] Master 127.22.2.190:35363 requested a full tablet report, sending...
I20260808 04:49:31.298177 17809 ts_manager.cc:194] Registered new tserver with Master: 53c642ea1bde49d3b2b095df4b667c69 (127.22.2.130:40759)
I20260808 04:49:31.298815 17809 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.130:51173
W20260808 04:49:31.412997 18266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:31.413282 18266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:31.413350 18266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:31.417558 18266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:31.417634 18266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:31.417719 18266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:49:31.422509 18266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:31.423547 18266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:31.425006 18266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:31.434836 18273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:31.434834 18272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:31.434827 18277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:31.435105 18266 server_base.cc:1034] running on GCE node
I20260808 04:49:31.435580 18266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:31.436234 18266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:31.437425 18266 hybrid_clock.cc:648] HybridClock initialized: now 1786164571437391 us; error 46 us; skew 500 ppm
I20260808 04:49:31.439258 18266 webserver.cc:460] Webserver started at http://127.22.2.131:33047/ using document root <none> and password file <none>
I20260808 04:49:31.439889 18266 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:31.439983 18266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:31.440225 18266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:31.442287 18266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "af2ea473396e4b5d93fcb4a9c5e783d1"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "41eced184f714e6f89b8c0d9de5b7687"
server_key_iv: "5ecc85614adf8dcd62deb19f7880c41a"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.442850 18266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "af2ea473396e4b5d93fcb4a9c5e783d1"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "41eced184f714e6f89b8c0d9de5b7687"
server_key_iv: "5ecc85614adf8dcd62deb19f7880c41a"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.446403 18266 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:49:31.448769 18285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.450029 18266 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:31.450160 18266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "af2ea473396e4b5d93fcb4a9c5e783d1"
format_stamp: "Formatted at 2026-08-08 04:49:31 on dist-test-slave-04bb"
server_key: "41eced184f714e6f89b8c0d9de5b7687"
server_key_iv: "5ecc85614adf8dcd62deb19f7880c41a"
server_key_version: "encryptionkey@0"
I20260808 04:49:31.450268 18266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:31.472636 18266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:31.473284 18266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:31.473484 18266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:31.474081 18266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:31.475044 18266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:31.475116 18266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.475185 18266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:31.475232 18266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:31.485206 18266 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:34887
I20260808 04:49:31.485245 18448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:34887 every 8 connection(s)
I20260808 04:49:31.486111 18266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:49:31.492094 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 18266
I20260808 04:49:31.492249 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:49:31.492571 22538 external_mini_cluster.cc:1428] Setting key 6bc6c732655b6445a392eaf3f4715cad
I20260808 04:49:31.496160 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.488081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:56955 (local address 127.22.2.190:35363)
0808 04:49:31.488232 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.488236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.489676 (+ 1440us) server_negotiation.cc:407] Connection header received
0808 04:49:31.491184 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.491188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.491249 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.491359 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:31.492259 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.492974 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.494266 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.494506 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.494600 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.495719 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:31.495722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:31.495724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:31.495738 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:31.495758 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:31.495766 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:31.495832 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.495977 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.496029 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:49:31.497064 18452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.488506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35363 (local address 127.22.2.131:56955)
0808 04:49:31.489479 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.489520 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:31.490961 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:31.491535 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:31.491542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:31.491624 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:31.492092 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.492104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.493106 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.493114 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.494125 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.494147 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.494666 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.494672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.494898 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.495465 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:31.495467 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:31.495506 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:31.495567 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:31.495846 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:31.495852 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:31.495857 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:49:31.495949 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 04:49:31.496198 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":133,"threads_started":1}
I20260808 04:49:31.498368 18449 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35363
I20260808 04:49:31.498654 18449 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:31.499218 18449 heartbeater.cc:511] Master 127.22.2.190:35363 requested a full tablet report, sending...
I20260808 04:49:31.500409 17809 ts_manager.cc:194] Registered new tserver with Master: af2ea473396e4b5d93fcb4a9c5e783d1 (127.22.2.131:34887)
I20260808 04:49:31.501077 17809 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.131:56955
I20260808 04:49:31.506557 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:49:31.513423 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.508705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.22.2.190:35363)
0808 04:49:31.508878 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.508886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.509053 (+ 167us) server_negotiation.cc:407] Connection header received
0808 04:49:31.509236 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.509240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.509291 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.509404 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:31.509759 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.510370 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.511370 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.511661 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.511814 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.512791 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:31.512796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:31.512799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:31.512814 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:31.512845 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:31.512856 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:31.513014 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.513204 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.513251 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 04:49:31.516494 17809 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:49:31.519608 17809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:49:31.535282 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.529124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33297 (local address 127.0.0.1:59762)
0808 04:49:31.530108 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.530124 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:31.530212 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:31.530850 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:31.530853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:31.530877 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:31.530975 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.530983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.532444 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.532449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.533548 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.533559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.534613 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.534618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.534742 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.534814 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 04:49:31.534941 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 04:49:31.535072 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":53,"thread_start_us":135,"threads_started":1}
I20260808 04:49:31.535326 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.528486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38756 (local address 127.22.2.131:34887)
0808 04:49:31.529441 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.529461 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.529493 (+ 32us) server_negotiation.cc:407] Connection header received
0808 04:49:31.529587 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.529594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.529839 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.530502 (+ 663us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:31.530683 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.531653 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.533929 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.534687 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.534852 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.535030 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.535083 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.535174 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":192,"threads_started":1}
I20260808 04:49:31.535291 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.528317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:34887 (local address 127.0.0.1:38756)
0808 04:49:31.528896 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.528950 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:31.529086 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:31.530070 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:31.530090 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:31.530120 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:31.530316 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.530329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.531791 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.531798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.533765 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.533779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.534854 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.534858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.534945 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.534967 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 04:49:31.535034 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 04:49:31.535108 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":175,"thread_start_us":97,"threads_started":1}
I20260808 04:49:31.535933 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.529648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.22.2.129:33297)
0808 04:49:31.530373 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.530387 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.530411 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:49:31.530486 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.530490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.530681 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.530845 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:31.531114 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.532129 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.533683 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.534454 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.534574 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.534781 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.535629 (+ 848us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.535740 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":162,"threads_started":1}
I20260808 04:49:31.536302 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.528872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:40759 (local address 127.0.0.1:45544)
0808 04:49:31.529413 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.529428 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:31.529517 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:31.530346 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:31.530351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:31.530373 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:31.530475 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.530482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.531924 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.531928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.533765 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.533779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.535150 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.535155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.535256 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.535284 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 04:49:31.535888 (+ 604us) client_negotiation.cc:240] Negotiation successful
0808 04:49:31.536150 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":255,"thread_start_us":127,"threads_started":1}
I20260808 04:49:31.537036 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.529086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.22.2.130:40759)
0808 04:49:31.529778 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.529792 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.529818 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:49:31.529896 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.529901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.530104 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.530279 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:31.530602 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.531792 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.533958 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.534721 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.534848 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.535060 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.536733 (+ 1673us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.536852 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":193,"threads_started":1}
I20260808 04:49:31.538594 18357 tablet_service.cc:1467] Processing CreateTablet for tablet 44246e64c1474689a2697d9ee0de5046 (DEFAULT_TABLE table=test-table [id=bc8ae01bde404caf8500bf09b5735122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:31.538950 17976 tablet_service.cc:1467] Processing CreateTablet for tablet 44246e64c1474689a2697d9ee0de5046 (DEFAULT_TABLE table=test-table [id=bc8ae01bde404caf8500bf09b5735122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:31.539657 18357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44246e64c1474689a2697d9ee0de5046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:31.540103 17976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44246e64c1474689a2697d9ee0de5046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:31.540011 18165 tablet_service.cc:1467] Processing CreateTablet for tablet 44246e64c1474689a2697d9ee0de5046 (DEFAULT_TABLE table=test-table [id=bc8ae01bde404caf8500bf09b5735122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:31.541139 18165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44246e64c1474689a2697d9ee0de5046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:31.546785 18470 tablet_bootstrap.cc:492] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: Bootstrap starting.
I20260808 04:49:31.548645 18473 tablet_bootstrap.cc:492] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: Bootstrap starting.
I20260808 04:49:31.549199 18470 tablet_bootstrap.cc:654] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:31.549220 18472 tablet_bootstrap.cc:492] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e: Bootstrap starting.
I20260808 04:49:31.550953 18472 tablet_bootstrap.cc:654] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:31.550985 18473 tablet_bootstrap.cc:654] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:31.551119 18470 log.cc:824] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:31.551651 18472 log.cc:824] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:31.551877 18473 log.cc:824] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:31.553496 18470 tablet_bootstrap.cc:492] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: No bootstrap required, opened a new log
I20260808 04:49:31.553601 18472 tablet_bootstrap.cc:492] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e: No bootstrap required, opened a new log
I20260808 04:49:31.553696 18470 ts_tablet_manager.cc:1397] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 04:49:31.553839 18472 ts_tablet_manager.cc:1397] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 04:49:31.554119 18473 tablet_bootstrap.cc:492] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: No bootstrap required, opened a new log
I20260808 04:49:31.554373 18473 ts_tablet_manager.cc:1397] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:49:31.557238 18472 raft_consensus.cc:348] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.557511 18472 raft_consensus.cc:374] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:31.557569 18472 raft_consensus.cc:729] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe05bfe20f84d699d3c80ebf999f03e, State: Initialized, Role: FOLLOWER
I20260808 04:49:31.558046 18472 consensus_queue.cc:260] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.558063 18470 raft_consensus.cc:348] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.558331 18470 raft_consensus.cc:374] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:31.558388 18470 raft_consensus.cc:729] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2ea473396e4b5d93fcb4a9c5e783d1, State: Initialized, Role: FOLLOWER
I20260808 04:49:31.558610 18473 raft_consensus.cc:348] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.558840 18473 raft_consensus.cc:374] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:31.558894 18473 raft_consensus.cc:729] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53c642ea1bde49d3b2b095df4b667c69, State: Initialized, Role: FOLLOWER
I20260808 04:49:31.558826 18470 consensus_queue.cc:260] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.559092 18472 ts_tablet_manager.cc:1428] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 04:49:31.559329 18473 consensus_queue.cc:260] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.559466 18069 heartbeater.cc:503] Master 127.22.2.190:35363 was elected leader, sending a full tablet report...
I20260808 04:49:31.559470 18449 heartbeater.cc:503] Master 127.22.2.190:35363 was elected leader, sending a full tablet report...
I20260808 04:49:31.559798 18470 ts_tablet_manager.cc:1428] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 04:49:31.560201 18473 ts_tablet_manager.cc:1428] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:49:31.560339 18263 heartbeater.cc:503] Master 127.22.2.190:35363 was elected leader, sending a full tablet report...
I20260808 04:49:31.561055 18478 raft_consensus.cc:482] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:31.561255 18478 raft_consensus.cc:504] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.563700 18478 leader_election.cc:288] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbe05bfe20f84d699d3c80ebf999f03e (127.22.2.129:33297), 53c642ea1bde49d3b2b095df4b667c69 (127.22.2.130:40759)
I20260808 04:49:31.566855 18452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.562633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33297 (local address 127.22.2.131:50343)
0808 04:49:31.562931 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.562945 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:31.563065 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:31.563431 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:31.563437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:31.563456 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:31.563573 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.563581 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.564609 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.564612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.565700 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.565709 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.566430 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.566434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.566533 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.566555 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 04:49:31.566655 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 04:49:31.566729 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260808 04:49:31.566855 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.562696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:50343 (local address 127.22.2.129:33297)
0808 04:49:31.562866 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.562871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.563021 (+ 150us) server_negotiation.cc:407] Connection header received
0808 04:49:31.563172 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.563177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.563223 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.563314 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:31.563707 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.564453 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.565825 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.566304 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.566436 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.566485 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.566681 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.566748 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 04:49:31.567518 18004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44246e64c1474689a2697d9ee0de5046" candidate_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" is_pre_election: true
I20260808 04:49:31.567723 18004 raft_consensus.cc:2454] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2ea473396e4b5d93fcb4a9c5e783d1 in term 0.
I20260808 04:49:31.568143 18289 leader_election.cc:302] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2ea473396e4b5d93fcb4a9c5e783d1, fbe05bfe20f84d699d3c80ebf999f03e; no voters:
I20260808 04:49:31.568393 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.564342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35371 (local address 127.22.2.130:40759)
0808 04:49:31.564482 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.564489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.564507 (+ 18us) server_negotiation.cc:407] Connection header received
0808 04:49:31.564589 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.564591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.564634 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.564708 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:31.565262 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.565931 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.567300 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.567820 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.567950 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.567998 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.568202 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.568279 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:49:31.568354 18478 raft_consensus.cc:2781] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:49:31.568449 18478 raft_consensus.cc:482] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:49:31.568497 18478 raft_consensus.cc:3037] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:31.569473 18478 raft_consensus.cc:504] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.569823 18478 leader_election.cc:288] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [CANDIDATE]: Term 1 election: Requested vote from peers fbe05bfe20f84d699d3c80ebf999f03e (127.22.2.129:33297), 53c642ea1bde49d3b2b095df4b667c69 (127.22.2.130:40759)
I20260808 04:49:31.570075 18481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.563877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:40759 (local address 127.22.2.131:35371)
0808 04:49:31.564117 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.564131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:31.564222 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:31.564827 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:31.564830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:31.564847 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:31.564988 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.564997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.566023 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.566089 (+ 66us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.566830 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:31.566839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.567963 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.567967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:31.568066 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.568098 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 04:49:31.568185 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 04:49:31.569934 (+ 1749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260808 04:49:31.570102 18004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44246e64c1474689a2697d9ee0de5046" candidate_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbe05bfe20f84d699d3c80ebf999f03e"
I20260808 04:49:31.570215 18004 raft_consensus.cc:3037] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:31.570693 18193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44246e64c1474689a2697d9ee0de5046" candidate_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53c642ea1bde49d3b2b095df4b667c69" is_pre_election: true
I20260808 04:49:31.570686 18191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44246e64c1474689a2697d9ee0de5046" candidate_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53c642ea1bde49d3b2b095df4b667c69"
I20260808 04:49:31.570911 18191 raft_consensus.cc:3037] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:31.571581 18004 raft_consensus.cc:2454] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2ea473396e4b5d93fcb4a9c5e783d1 in term 1.
I20260808 04:49:31.571871 18289 leader_election.cc:302] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2ea473396e4b5d93fcb4a9c5e783d1, fbe05bfe20f84d699d3c80ebf999f03e; no voters:
I20260808 04:49:31.571923 18191 raft_consensus.cc:2454] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2ea473396e4b5d93fcb4a9c5e783d1 in term 1.
I20260808 04:49:31.572090 18478 raft_consensus.cc:2781] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:31.572196 18478 raft_consensus.cc:686] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 1 LEADER]: Becoming Leader. State: Replica: af2ea473396e4b5d93fcb4a9c5e783d1, State: Running, Role: LEADER
I20260808 04:49:31.572482 18478 consensus_queue.cc:237] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } }
I20260808 04:49:31.575508 17808 catalog_manager.cc:5495] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to af2ea473396e4b5d93fcb4a9c5e783d1 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2ea473396e4b5d93fcb4a9c5e783d1" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 } health_report { overall_health: UNKNOWN } } }
W20260808 04:49:31.578576 18070 tablet.cc:2369] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:31.597366 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.593611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.22.2.131:34887)
0808 04:49:31.593731 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:31.593738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:31.593754 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:49:31.593948 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:31.593951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:31.594006 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:31.594117 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:49:31.594378 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.594878 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.595775 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:31.595995 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:31.596114 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:31.596446 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:49:31.596915 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:49:31.597080 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:31.597190 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 04:49:31.597225 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:49:31.603327 18191 raft_consensus.cc:1261] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [term 1 FOLLOWER]: Refusing update from remote peer af2ea473396e4b5d93fcb4a9c5e783d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:31.603389 18004 raft_consensus.cc:1261] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e [term 1 FOLLOWER]: Refusing update from remote peer af2ea473396e4b5d93fcb4a9c5e783d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:31.604033 18480 consensus_queue.cc:1035] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53c642ea1bde49d3b2b095df4b667c69" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:31.604282 18478 consensus_queue.cc:1035] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbe05bfe20f84d699d3c80ebf999f03e" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:31.615474 18488 mvcc.cc:205] Tried to move back new op lower bound from 7316130085279776768 to 7316130085163368448. Current Snapshot: MvccSnapshot[applied={T|T < 7316130085279776768}]
I20260808 04:49:31.621168 17808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:31.621308 17808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:31.623569 17808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c35d25257b444a7964bac151d325d34: Sending DeleteTablet for 3 replicas of tablet 44246e64c1474689a2697d9ee0de5046
I20260808 04:49:31.624377 18357 tablet_service.cc:1514] Processing DeleteTablet for tablet 44246e64c1474689a2697d9ee0de5046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:31 UTC) from {username='slave'} at 127.0.0.1:38756
I20260808 04:49:31.624444 18165 tablet_service.cc:1514] Processing DeleteTablet for tablet 44246e64c1474689a2697d9ee0de5046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:31 UTC) from {username='slave'} at 127.0.0.1:45544
I20260808 04:49:31.624913 17976 tablet_service.cc:1514] Processing DeleteTablet for tablet 44246e64c1474689a2697d9ee0de5046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:31 UTC) from {username='slave'} at 127.0.0.1:59762
I20260808 04:49:31.625375 18505 tablet_replica.cc:330] T 44246e64c1474689a2697d9ee0de5046 P fbe05bfe20f84d699d3c80ebf999f03e: stopping tablet replica
I20260808 04:49:31.625375 18504 tablet_replica.cc:330] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: stopping tablet replica
I20260808 04:49:31.625473 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 17899
I20260808 04:49:31.625654 18504 raft_consensus.cc:2229] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:31.625895 18504 raft_consensus.cc:2258] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:31.626116 18503 tablet_replica.cc:330] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: stopping tablet replica
I20260808 04:49:31.626837 18503 raft_consensus.cc:2229] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:49:31.627082 18503 raft_consensus.cc:2258] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:31.631139 18503 ts_tablet_manager.cc:1905] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:31.632046 18504 ts_tablet_manager.cc:1905] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:31.634069 18503 ts_tablet_manager.cc:1918] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:31.634178 18503 log.cc:1196] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/44246e64c1474689a2697d9ee0de5046
I20260808 04:49:31.634487 18503 ts_tablet_manager.cc:1939] T 44246e64c1474689a2697d9ee0de5046 P af2ea473396e4b5d93fcb4a9c5e783d1: Deleting consensus metadata
I20260808 04:49:31.635000 18504 ts_tablet_manager.cc:1918] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:31.635084 18504 log.cc:1196] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/44246e64c1474689a2697d9ee0de5046
I20260808 04:49:31.635392 18504 ts_tablet_manager.cc:1939] T 44246e64c1474689a2697d9ee0de5046 P 53c642ea1bde49d3b2b095df4b667c69: Deleting consensus metadata
I20260808 04:49:31.635809 17789 catalog_manager.cc:4841] TS af2ea473396e4b5d93fcb4a9c5e783d1 (127.22.2.131:34887): tablet 44246e64c1474689a2697d9ee0de5046 (table test-table [id=bc8ae01bde404caf8500bf09b5735122]) successfully deleted
I20260808 04:49:31.636507 17791 catalog_manager.cc:4841] TS 53c642ea1bde49d3b2b095df4b667c69 (127.22.2.130:40759): tablet 44246e64c1474689a2697d9ee0de5046 (table test-table [id=bc8ae01bde404caf8500bf09b5735122]) successfully deleted
W20260808 04:49:31.641031 17789 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:33297 (error 108)
I20260808 04:49:31.641907 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 18073
I20260808 04:49:31.642215 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:31.641891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33297 (local address 127.0.0.1:59770)
0808 04:49:31.642053 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:31.642125 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 04:49:31.648077 17789 catalog_manager.cc:4584] TS fbe05bfe20f84d699d3c80ebf999f03e (127.22.2.129:33297): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44246e64c1474689a2697d9ee0de5046: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33297: connect: Connection refused (error 111)
I20260808 04:49:31.654160 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 18266
I20260808 04:49:31.667419 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 17764
2026-08-08T04:49:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1064 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:31 dist-test-slave-04bb krb5kdc[18509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:31 dist-test-slave-04bb krb5kdc[18509](info): set up 2 sockets
Aug 08 04:49:31 dist-test-slave-04bb krb5kdc[18509](info): commencing operation
krb5kdc: starting...
W20260808 04:49:32.914767 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:49:33.048851 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36671
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:36671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:33.162405 18531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:33.162727 18531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:33.162812 18531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:33.167454 18531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:33.167553 18531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:33.167589 18531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:33.167630 18531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:33.167665 18531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:33.173149 18531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:36671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:33.174258 18531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:33.175326 18531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:33.184160 18543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:33.184160 18541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:33.184271 18540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:33.184441 18531 server_base.cc:1034] running on GCE node
I20260808 04:49:33.185109 18531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:33.186199 18531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:33.187386 18531 hybrid_clock.cc:648] HybridClock initialized: now 1786164573187363 us; error 33 us; skew 500 ppm
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:33.190243 18531 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:49:33.191166 18531 webserver.cc:460] Webserver started at http://127.22.2.190:42081/ using document root <none> and password file <none>
I20260808 04:49:33.191700 18531 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:33.191757 18531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:33.191920 18531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:33.193981 18531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "491f16b12f924288b4f6bf5fb049d566"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "89b293f67839f54470a93078adda7ef0"
server_key_iv: "0248c969dd41f42814fd16557c43f274"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.194497 18531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "491f16b12f924288b4f6bf5fb049d566"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "89b293f67839f54470a93078adda7ef0"
server_key_iv: "0248c969dd41f42814fd16557c43f274"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.198240 18531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 04:49:33.200774 18553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.202139 18531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:49:33.202279 18531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "491f16b12f924288b4f6bf5fb049d566"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "89b293f67839f54470a93078adda7ef0"
server_key_iv: "0248c969dd41f42814fd16557c43f274"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.202392 18531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:33.220291 18531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:33.223379 18531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:33.223596 18531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:33.231187 18531 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:36671
I20260808 04:49:33.231217 18620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:36671 every 8 connection(s)
I20260808 04:49:33.232148 18531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:33.235060 18622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:33.236135 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 18531
I20260808 04:49:33.236281 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:33.236677 22538 external_mini_cluster.cc:1428] Setting key a398b9dc5213df6e5a831a5287f054da
I20260808 04:49:33.241389 18622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566: Bootstrap starting.
I20260808 04:49:33.243541 18622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:33.244477 18622 log.cc:824] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:33.246579 18622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566: No bootstrap required, opened a new log
I20260808 04:49:33.250205 18622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f16b12f924288b4f6bf5fb049d566" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36671 } }
I20260808 04:49:33.250439 18622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:33.250504 18622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491f16b12f924288b4f6bf5fb049d566, State: Initialized, Role: FOLLOWER
I20260808 04:49:33.251015 18622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f16b12f924288b4f6bf5fb049d566" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36671 } }
I20260808 04:49:33.251190 18622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:33.251272 18622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:33.251392 18622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:33.251982 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:33.238224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.22.2.190:36671)
0808 04:49:33.238669 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:33.238687 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:33.238729 (+ 42us) server_negotiation.cc:407] Connection header received
0808 04:49:33.239409 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:33.239428 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:33.239770 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:33.240131 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:33.241062 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.242174 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.243122 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.243663 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.243828 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:33.246729 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:33.246735 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:33.246751 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:33.246779 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:33.248676 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.249161 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.249169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.249180 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.249352 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.249671 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.249675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.249677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.250082 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:33.250341 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:33.250621 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 04:49:33.250979 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":123,"threads_started":1}
I20260808 04:49:33.252470 18622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f16b12f924288b4f6bf5fb049d566" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36671 } }
I20260808 04:49:33.252840 18622 leader_election.cc:302] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 491f16b12f924288b4f6bf5fb049d566; no voters:
I20260808 04:49:33.253168 18622 leader_election.cc:288] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:33.253283 18629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:33.253507 18629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [term 1 LEADER]: Becoming Leader. State: Replica: 491f16b12f924288b4f6bf5fb049d566, State: Running, Role: LEADER
I20260808 04:49:33.253832 18629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f16b12f924288b4f6bf5fb049d566" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36671 } }
I20260808 04:49:33.254438 18622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:33.255003 18633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 491f16b12f924288b4f6bf5fb049d566. Latest consensus state: current_term: 1 leader_uuid: "491f16b12f924288b4f6bf5fb049d566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f16b12f924288b4f6bf5fb049d566" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36671 } } }
I20260808 04:49:33.255139 18633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:33.255493 18630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "491f16b12f924288b4f6bf5fb049d566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f16b12f924288b4f6bf5fb049d566" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36671 } } }
I20260808 04:49:33.255595 18630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:33.257051 18638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:33.259568 18638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:33.265360 18638 catalog_manager.cc:1347] Generated new cluster ID: ab09da2528454e8ebb5842ceff49493f
I20260808 04:49:33.265427 18638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:33.282655 18638 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:33.283573 18638 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:33.294731 18638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566: Generated new TSK 0
I20260808 04:49:33.295463 18638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 04:49:33.332386 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36671
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:33.452201 18666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:33.452442 18666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:33.452483 18666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:33.456629 18666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:33.456681 18666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:33.456779 18666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:49:33.462044 18666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:33.462970 18666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:33.463938 18666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:33.473695 18673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:33.473728 18674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:33.473709 18676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:33.473881 18666 server_base.cc:1034] running on GCE node
I20260808 04:49:33.474385 18666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:33.475106 18666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:33.476305 18666 hybrid_clock.cc:648] HybridClock initialized: now 1786164573476274 us; error 43 us; skew 500 ppm
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:33.479113 18666 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 04:49:33.479980 18666 webserver.cc:460] Webserver started at http://127.22.2.129:41521/ using document root <none> and password file <none>
I20260808 04:49:33.480509 18666 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:33.480640 18666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:33.480824 18666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:33.482782 18666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "4548d6804d8a42a5a7e9e88f5267fcfe"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "7977e13d9be242c367ff58d8a0bb11da"
server_key_iv: "f1a75360bbd7ac8e5ddc34fa786049da"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.483319 18666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "4548d6804d8a42a5a7e9e88f5267fcfe"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "7977e13d9be242c367ff58d8a0bb11da"
server_key_iv: "f1a75360bbd7ac8e5ddc34fa786049da"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.486990 18666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 04:49:33.489526 18684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.490720 18666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:49:33.490857 18666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "4548d6804d8a42a5a7e9e88f5267fcfe"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "7977e13d9be242c367ff58d8a0bb11da"
server_key_iv: "f1a75360bbd7ac8e5ddc34fa786049da"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.490960 18666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:33.504667 18666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:33.507787 18666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:33.507992 18666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:33.508615 18666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:33.509433 18666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:33.509479 18666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.509546 18666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:33.509578 18666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.517920 18666 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:39869
I20260808 04:49:33.517973 18854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:39869 every 8 connection(s)
I20260808 04:49:33.518826 18666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:49:33.519385 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 18666
I20260808 04:49:33.519519 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:49:33.519807 22538 external_mini_cluster.cc:1428] Setting key 535dcb17b1c868e94dd572f28a913bf0
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 04:49:33.533519 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:33.520830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:51381 (local address 127.22.2.190:36671)
0808 04:49:33.520957 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:33.520961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:33.521910 (+ 949us) server_negotiation.cc:407] Connection header received
0808 04:49:33.523214 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:33.523225 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:33.523284 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:33.523362 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:33.524499 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.525319 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.526507 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.526921 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.527040 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:33.529564 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:33.529568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:33.529571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:33.529600 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:33.531239 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.531747 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.531754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.531758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.531851 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.532175 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.532178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.532179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.532378 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:33.532456 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:33.533284 (+ 828us) server_negotiation.cc:299] Negotiation successful
0808 04:49:33.533391 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 04:49:33.534578 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:33.521251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36671 (local address 127.22.2.129:51381)
0808 04:49:33.521765 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:33.521807 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:33.523006 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:33.523532 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:33.523539 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:33.523889 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:33.524347 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:33.524360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.525455 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.525458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:33.526329 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:33.526339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.527079 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.527082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:33.527219 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:33.527872 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:33.527889 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:33.529380 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:33.531362 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:33.531367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:33.531379 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:33.531623 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:33.531946 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:33.531949 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:33.531950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:33.532086 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:33.532471 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:33.532476 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:33.533044 (+ 568us) client_negotiation.cc:769] Sending connection context
0808 04:49:33.533211 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 04:49:33.533501 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":156,"threads_started":1}
I20260808 04:49:33.535897 18855 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36671
I20260808 04:49:33.536170 18855 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:33.536768 18855 heartbeater.cc:511] Master 127.22.2.190:36671 requested a full tablet report, sending...
I20260808 04:49:33.538552 18578 ts_manager.cc:194] Registered new tserver with Master: 4548d6804d8a42a5a7e9e88f5267fcfe (127.22.2.129:39869)
I20260808 04:49:33.540347 18578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:51381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 04:49:33.551411 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36671
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:33.665954 18865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:33.666189 18865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:33.666229 18865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:33.670087 18865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:33.670131 18865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:33.670185 18865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:49:33.675102 18865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:33.675992 18865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:33.676940 18865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:33.686697 18873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:33.686718 18876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:33.686697 18872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:33.687007 18865 server_base.cc:1034] running on GCE node
I20260808 04:49:33.687399 18865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:33.688105 18865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:33.689291 18865 hybrid_clock.cc:648] HybridClock initialized: now 1786164573689270 us; error 33 us; skew 500 ppm
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:33.691987 18865 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 04:49:33.692975 18865 webserver.cc:460] Webserver started at http://127.22.2.130:44273/ using document root <none> and password file <none>
I20260808 04:49:33.693558 18865 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:33.693652 18865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:33.693871 18865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:33.695897 18865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "2ee9c42c2c1d473097d659dea48424c4"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "5897efd00422a7ba8acff61e8fac510a"
server_key_iv: "37dd6d45d4578a07c66db1a1d22c8640"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.696466 18865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "2ee9c42c2c1d473097d659dea48424c4"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "5897efd00422a7ba8acff61e8fac510a"
server_key_iv: "37dd6d45d4578a07c66db1a1d22c8640"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.700073 18865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:49:33.702696 18884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.703913 18865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:33.704049 18865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "2ee9c42c2c1d473097d659dea48424c4"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "5897efd00422a7ba8acff61e8fac510a"
server_key_iv: "37dd6d45d4578a07c66db1a1d22c8640"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.704159 18865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:33.720398 18865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:33.723335 18865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:33.723531 18865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:33.724094 18865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:33.724979 18865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:33.725050 18865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.725124 18865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:33.725168 18865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.734179 18865 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41409
I20260808 04:49:33.734257 19049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41409 every 8 connection(s)
I20260808 04:49:33.735067 18865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:49:33.738349 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 18865
I20260808 04:49:33.738498 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:49:33.738795 22538 external_mini_cluster.cc:1428] Setting key 72bdc5fa2e088d90a0e5dc34a5867b20
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 04:49:33.750806 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:33.737161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:44959 (local address 127.22.2.190:36671)
0808 04:49:33.737391 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:33.737395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:33.738229 (+ 834us) server_negotiation.cc:407] Connection header received
0808 04:49:33.740044 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:33.740047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:33.740092 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:33.740194 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:33.741641 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.742214 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.743484 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.743786 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.743918 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:33.746530 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:33.746537 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:33.746544 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:33.746584 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:33.748449 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.748997 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.749003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.749008 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.749115 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.749492 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.749498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.749502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.749785 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:33.749917 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:33.750426 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 04:49:33.750553 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260808 04:49:33.751593 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:33.737533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36671 (local address 127.22.2.130:44959)
0808 04:49:33.738023 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:33.738091 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:33.739796 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:33.740435 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:33.740445 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:33.740939 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:33.741466 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:33.741481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.742363 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.742367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:33.743343 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:33.743353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.743935 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.743939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:33.744099 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:33.744793 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:33.744810 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:33.746343 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:33.748615 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:33.748619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:33.748632 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:33.748883 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:33.749243 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:33.749245 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:33.749246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:33.749379 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:33.749901 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:33.749906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:33.750128 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 04:49:33.750320 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 04:49:33.750566 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":125,"threads_started":1}
I20260808 04:49:33.753036 19050 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36671
I20260808 04:49:33.753306 19050 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 04:49:33.753961 19050 heartbeater.cc:511] Master 127.22.2.190:36671 requested a full tablet report, sending...
I20260808 04:49:33.755259 18578 ts_manager.cc:194] Registered new tserver with Master: 2ee9c42c2c1d473097d659dea48424c4 (127.22.2.130:41409)
I20260808 04:49:33.755960 18578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:44959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 04:49:33.769948 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36671
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:33.886116 19056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:33.886365 19056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:33.886408 19056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:33.890620 19056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:33.890677 19056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:33.890743 19056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:49:33.896136 19056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:36671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:33.897223 19056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:33.898211 19056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:33.908329 19063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:33.908377 19067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:33.908510 19056 server_base.cc:1034] running on GCE node
W20260808 04:49:33.908380 19064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:33.909029 19056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:33.909668 19056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:33.910858 19056 hybrid_clock.cc:648] HybridClock initialized: now 1786164573910849 us; error 34 us; skew 500 ppm
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:33.913767 19056 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 04:49:33.914566 19056 webserver.cc:460] Webserver started at http://127.22.2.131:44669/ using document root <none> and password file <none>
I20260808 04:49:33.915082 19056 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:33.915141 19056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:33.915385 19056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:33.917448 19056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "ef061c35c883470686984d0df078eecd"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "1b2d44801e296d4c28395a985e1951a3"
server_key_iv: "eb138370164f6c0c258fdd58c30d6d61"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.917956 19056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "ef061c35c883470686984d0df078eecd"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "1b2d44801e296d4c28395a985e1951a3"
server_key_iv: "eb138370164f6c0c258fdd58c30d6d61"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.921595 19056 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:49:33.923945 19079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.925195 19056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:49:33.925324 19056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "ef061c35c883470686984d0df078eecd"
format_stamp: "Formatted at 2026-08-08 04:49:33 on dist-test-slave-04bb"
server_key: "1b2d44801e296d4c28395a985e1951a3"
server_key_iv: "eb138370164f6c0c258fdd58c30d6d61"
server_key_version: "encryptionkey@0"
I20260808 04:49:33.925462 19056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:33.943801 19056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:33.947190 19056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:33.947410 19056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:33.948051 19056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:33.949004 19056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:33.949052 19056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.949115 19056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:33.949149 19056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:33.958551 19056 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:40819
I20260808 04:49:33.958593 19237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:40819 every 8 connection(s)
I20260808 04:49:33.959429 19056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:49:33.967321 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 19056
I20260808 04:49:33.967483 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:49:33.967859 22538 external_mini_cluster.cc:1428] Setting key 31076eaa34034766021370b274337b89
Aug 08 04:49:33 dist-test-slave-04bb krb5kdc[18509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164573, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:33.974184 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:33.961392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:39215 (local address 127.22.2.190:36671)
0808 04:49:33.961579 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:33.961584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:33.962438 (+ 854us) server_negotiation.cc:407] Connection header received
0808 04:49:33.963688 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:33.963691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:33.963742 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:33.963828 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:33.965034 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.965832 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.966935 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.967392 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.967529 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:33.970500 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:33.970505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:33.970508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:33.970540 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:33.971933 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.972496 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.972502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.972504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.972628 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.973001 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.973006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.973008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.973220 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:33.973348 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:33.973861 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 04:49:33.973989 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 04:49:33.974958 19240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:33.961814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:36671 (local address 127.22.2.131:39215)
0808 04:49:33.962272 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:33.962318 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:33.963459 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:33.963992 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:33.963999 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:33.964336 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:33.964878 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:33.964891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.965956 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.965959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:33.966811 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:33.966820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.967608 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.967611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:33.967747 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:33.968476 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:49:33.968492 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:33.970332 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:33.972063 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:33.972069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:33.972084 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:33.972379 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:33.972738 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:49:33.972743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:49:33.972745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:49:33.972893 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:49:33.973345 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:33.973352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:33.973599 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 04:49:33.973808 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 04:49:33.974133 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260808 04:49:33.976270 19238 heartbeater.cc:347] Connected to a master server at 127.22.2.190:36671
I20260808 04:49:33.976578 19238 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:33.977089 19238 heartbeater.cc:511] Master 127.22.2.190:36671 requested a full tablet report, sending...
I20260808 04:49:33.978186 18578 ts_manager.cc:194] Registered new tserver with Master: ef061c35c883470686984d0df078eecd (127.22.2.131:40819)
I20260808 04:49:33.978859 18578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:39215
I20260808 04:49:33.981954 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:49:33.991338 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:33.983961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.22.2.190:36671)
0808 04:49:33.984091 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:33.984095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:33.984241 (+ 146us) server_negotiation.cc:407] Connection header received
0808 04:49:33.984351 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:33.984353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:33.984388 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:33.984457 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:33.985206 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.985712 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.986553 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:33.986776 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:33.986858 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:33.988320 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:33.988326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:33.988329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:33.988376 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:33.989774 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.990203 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.990206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.990207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.990290 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:33.990568 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:33.990570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:33.990572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:33.990809 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:33.990914 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:33.991126 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 04:49:33.991222 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:49:33.994366 18578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:49:33.997927 18578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:49:34.012950 19260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.006890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:39869 (local address 127.0.0.1:47918)
0808 04:49:34.007329 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:34.007369 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:34.007507 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:34.008469 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:34.008474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:34.008496 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:34.008668 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.008677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.010250 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.010253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.011475 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.011487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.012334 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.012338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.012452 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.012490 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 04:49:34.012644 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 04:49:34.012794 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":61,"spinlock_wait_cycles":7552,"thread_start_us":108,"threads_started":1}
I20260808 04:49:34.013082 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.006652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60852 (local address 127.22.2.130:41409)
0808 04:49:34.007811 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.007839 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.008099 (+ 260us) server_negotiation.cc:407] Connection header received
0808 04:49:34.008174 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.008179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.008325 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.008479 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:34.008876 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.010105 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.011510 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.012361 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.012469 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.012630 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.012822 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.012929 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1050,"thread_start_us":139,"threads_started":1}
I20260808 04:49:34.013154 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.006826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40819 (local address 127.0.0.1:39614)
0808 04:49:34.007318 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:34.007370 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:34.007507 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:34.008732 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:34.008735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:34.008812 (+ 77us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:34.008910 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.008915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.009984 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.009988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.011475 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.011487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.012704 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.012712 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.012851 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.012874 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 04:49:34.012943 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 04:49:34.013008 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":67,"spinlock_wait_cycles":5248,"thread_start_us":82,"threads_started":1}
I20260808 04:49:34.013556 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.006463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:41409 (local address 127.0.0.1:60852)
0808 04:49:34.007915 (+ 1452us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:34.007928 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:34.008012 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:34.008471 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:34.008476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:34.008570 (+ 94us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:34.008721 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.008730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.010289 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.010293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.011390 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.011399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.012585 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.012589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.012685 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.012705 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 04:49:34.012777 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 04:49:34.013429 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":611,"thread_start_us":73,"threads_started":1}
I20260808 04:49:34.013615 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.007366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.22.2.131:40819)
0808 04:49:34.008034 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.008052 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.008122 (+ 70us) server_negotiation.cc:407] Connection header received
0808 04:49:34.008211 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.008217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.008428 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.009032 (+ 604us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:34.009148 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.009842 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.011607 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.012361 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.012469 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.012630 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.013405 (+ 775us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.013483 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":162,"threads_started":1}
I20260808 04:49:34.014605 19263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.007524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.22.2.129:39869)
0808 04:49:34.007811 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.007841 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.007866 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:49:34.007950 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.007958 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.008279 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.008824 (+ 545us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:34.008998 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.010111 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.011647 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.012203 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.012333 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.012483 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.012872 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.012970 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260808 04:49:34.016700 18953 tablet_service.cc:1467] Processing CreateTablet for tablet b097c64c553b4445bd232b13ebebd76f (DEFAULT_TABLE table=test-table [id=1f219438aaf24e64bcbdf200570adc6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:34.016700 19145 tablet_service.cc:1467] Processing CreateTablet for tablet b097c64c553b4445bd232b13ebebd76f (DEFAULT_TABLE table=test-table [id=1f219438aaf24e64bcbdf200570adc6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:34.016907 18751 tablet_service.cc:1467] Processing CreateTablet for tablet b097c64c553b4445bd232b13ebebd76f (DEFAULT_TABLE table=test-table [id=1f219438aaf24e64bcbdf200570adc6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:34.017848 18953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b097c64c553b4445bd232b13ebebd76f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:34.017858 19145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b097c64c553b4445bd232b13ebebd76f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:34.017923 18751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b097c64c553b4445bd232b13ebebd76f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:34.024240 19265 tablet_bootstrap.cc:492] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe: Bootstrap starting.
I20260808 04:49:34.025832 19268 tablet_bootstrap.cc:492] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Bootstrap starting.
I20260808 04:49:34.025832 19267 tablet_bootstrap.cc:492] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: Bootstrap starting.
I20260808 04:49:34.026589 19265 tablet_bootstrap.cc:654] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:34.027685 19265 log.cc:824] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:34.028008 19268 tablet_bootstrap.cc:654] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:34.028471 19267 tablet_bootstrap.cc:654] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:34.029381 19268 log.cc:824] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:34.029382 19267 log.cc:824] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:34.029888 19265 tablet_bootstrap.cc:492] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe: No bootstrap required, opened a new log
I20260808 04:49:34.030100 19265 ts_tablet_manager.cc:1397] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:49:34.031421 19268 tablet_bootstrap.cc:492] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: No bootstrap required, opened a new log
I20260808 04:49:34.031421 19267 tablet_bootstrap.cc:492] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: No bootstrap required, opened a new log
I20260808 04:49:34.031605 19267 ts_tablet_manager.cc:1397] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 04:49:34.031651 19268 ts_tablet_manager.cc:1397] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:49:34.033144 19265 raft_consensus.cc:348] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.033382 19265 raft_consensus.cc:374] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:34.033437 19265 raft_consensus.cc:729] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4548d6804d8a42a5a7e9e88f5267fcfe, State: Initialized, Role: FOLLOWER
I20260808 04:49:34.033890 19265 consensus_queue.cc:260] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.034561 18855 heartbeater.cc:503] Master 127.22.2.190:36671 was elected leader, sending a full tablet report...
I20260808 04:49:34.034929 19265 ts_tablet_manager.cc:1428] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:49:34.035504 19268 raft_consensus.cc:348] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.035504 19267 raft_consensus.cc:348] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.035785 19267 raft_consensus.cc:374] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:34.035785 19268 raft_consensus.cc:374] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:34.035846 19268 raft_consensus.cc:729] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef061c35c883470686984d0df078eecd, State: Initialized, Role: FOLLOWER
I20260808 04:49:34.035856 19267 raft_consensus.cc:729] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee9c42c2c1d473097d659dea48424c4, State: Initialized, Role: FOLLOWER
I20260808 04:49:34.036281 19267 consensus_queue.cc:260] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.036279 19268 consensus_queue.cc:260] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.036928 19050 heartbeater.cc:503] Master 127.22.2.190:36671 was elected leader, sending a full tablet report...
I20260808 04:49:34.037191 19238 heartbeater.cc:503] Master 127.22.2.190:36671 was elected leader, sending a full tablet report...
I20260808 04:49:34.038196 19268 ts_tablet_manager.cc:1428] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260808 04:49:34.038568 19267 ts_tablet_manager.cc:1428] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260808 04:49:34.169971 19275 raft_consensus.cc:482] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:34.170226 19275 raft_consensus.cc:504] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.171394 19275 leader_election.cc:288] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef061c35c883470686984d0df078eecd (127.22.2.131:40819), 4548d6804d8a42a5a7e9e88f5267fcfe (127.22.2.129:39869)
I20260808 04:49:34.175101 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.171758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:58853 (local address 127.22.2.131:40819)
0808 04:49:34.171905 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.171910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.171926 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:49:34.172017 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.172020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.172072 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.172173 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:34.172462 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.173030 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.174104 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.174528 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.174675 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.174733 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.174910 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.174979 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 04:49:34.175101 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.171656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40819 (local address 127.22.2.130:58853)
0808 04:49:34.171798 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:34.171816 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:34.171919 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:34.172188 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:34.172191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:34.172212 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:34.172337 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.172343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.173187 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.173191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.173965 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.173975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.174658 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.174661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.174771 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.174800 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 04:49:34.174892 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 04:49:34.174964 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 04:49:34.175530 19277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.171663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:39869 (local address 127.22.2.130:46425)
0808 04:49:34.172323 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:34.172334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:34.172464 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:34.172778 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:34.172781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:34.172797 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:34.172920 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.172926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.173631 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.173636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.174410 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.174417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.175061 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.175065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.175173 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.175212 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 04:49:34.175331 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 04:49:34.175395 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":275,"thread_start_us":167,"threads_started":1}
I20260808 04:49:34.175599 19263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.171795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:46425 (local address 127.22.2.129:39869)
0808 04:49:34.171925 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.171929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.172434 (+ 505us) server_negotiation.cc:407] Connection header received
0808 04:49:34.172627 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.172630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.172674 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.172767 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:34.173037 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.173517 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.174544 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.174936 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.175066 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.175116 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.175390 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.175471 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 04:49:34.175810 19181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b097c64c553b4445bd232b13ebebd76f" candidate_uuid: "2ee9c42c2c1d473097d659dea48424c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef061c35c883470686984d0df078eecd" is_pre_election: true
I20260808 04:49:34.176160 19181 raft_consensus.cc:2454] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ee9c42c2c1d473097d659dea48424c4 in term 0.
I20260808 04:49:34.176267 18776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b097c64c553b4445bd232b13ebebd76f" candidate_uuid: "2ee9c42c2c1d473097d659dea48424c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" is_pre_election: true
I20260808 04:49:34.176627 18776 raft_consensus.cc:2454] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ee9c42c2c1d473097d659dea48424c4 in term 0.
I20260808 04:49:34.176687 18885 leader_election.cc:302] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ee9c42c2c1d473097d659dea48424c4, ef061c35c883470686984d0df078eecd; no voters:
I20260808 04:49:34.177038 19275 raft_consensus.cc:2781] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:49:34.177153 19275 raft_consensus.cc:482] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:49:34.177208 19275 raft_consensus.cc:3037] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:34.178550 19275 raft_consensus.cc:504] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.178897 19275 leader_election.cc:288] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [CANDIDATE]: Term 1 election: Requested vote from peers ef061c35c883470686984d0df078eecd (127.22.2.131:40819), 4548d6804d8a42a5a7e9e88f5267fcfe (127.22.2.129:39869)
I20260808 04:49:34.179292 18776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b097c64c553b4445bd232b13ebebd76f" candidate_uuid: "2ee9c42c2c1d473097d659dea48424c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe"
I20260808 04:49:34.179313 19181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b097c64c553b4445bd232b13ebebd76f" candidate_uuid: "2ee9c42c2c1d473097d659dea48424c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef061c35c883470686984d0df078eecd"
I20260808 04:49:34.179416 18776 raft_consensus.cc:3037] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:34.179445 19181 raft_consensus.cc:3037] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:34.180480 18776 raft_consensus.cc:2454] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ee9c42c2c1d473097d659dea48424c4 in term 1.
I20260808 04:49:34.180490 19181 raft_consensus.cc:2454] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ee9c42c2c1d473097d659dea48424c4 in term 1.
I20260808 04:49:34.180833 18886 leader_election.cc:302] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ee9c42c2c1d473097d659dea48424c4, 4548d6804d8a42a5a7e9e88f5267fcfe; no voters:
I20260808 04:49:34.181020 19275 raft_consensus.cc:2781] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:34.181223 19275 raft_consensus.cc:686] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 1 LEADER]: Becoming Leader. State: Replica: 2ee9c42c2c1d473097d659dea48424c4, State: Running, Role: LEADER
I20260808 04:49:34.181597 19275 consensus_queue.cc:237] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } }
I20260808 04:49:34.185187 18577 catalog_manager.cc:5495] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ee9c42c2c1d473097d659dea48424c4 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "2ee9c42c2c1d473097d659dea48424c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee9c42c2c1d473097d659dea48424c4" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 41409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 } health_report { overall_health: UNKNOWN } } }
I20260808 04:49:34.202140 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.198349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.22.2.130:41409)
0808 04:49:34.198495 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.198500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.198514 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:49:34.198674 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.198677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.198738 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.198855 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:49:34.199097 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.199603 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.200506 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.200880 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.200994 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.201313 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:49:34.201722 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:49:34.201859 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.201923 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.201978 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:49:34.208192 18776 raft_consensus.cc:1261] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe [term 1 FOLLOWER]: Refusing update from remote peer 2ee9c42c2c1d473097d659dea48424c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:34.208191 19181 raft_consensus.cc:1261] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 1 FOLLOWER]: Refusing update from remote peer 2ee9c42c2c1d473097d659dea48424c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:34.208945 19275 consensus_queue.cc:1035] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef061c35c883470686984d0df078eecd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:34.209184 19278 consensus_queue.cc:1035] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4548d6804d8a42a5a7e9e88f5267fcfe" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 39869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 04:49:34.210345 19239 tablet.cc:2369] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:34.215619 19281 mvcc.cc:205] Tried to move back new op lower bound from 7316130095948816384 to 7316130095850737664. Current Snapshot: MvccSnapshot[applied={T|T < 7316130095948816384}]
I20260808 04:49:34.215865 19283 mvcc.cc:205] Tried to move back new op lower bound from 7316130095948816384 to 7316130095850737664. Current Snapshot: MvccSnapshot[applied={T|T < 7316130095948816384}]
I20260808 04:49:34.224224 18577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:34.224391 18577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:34.226857 18577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 491f16b12f924288b4f6bf5fb049d566: Sending DeleteTablet for 3 replicas of tablet b097c64c553b4445bd232b13ebebd76f
I20260808 04:49:34.227598 18953 tablet_service.cc:1514] Processing DeleteTablet for tablet b097c64c553b4445bd232b13ebebd76f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:34 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60852
I20260808 04:49:34.227620 19145 tablet_service.cc:1514] Processing DeleteTablet for tablet b097c64c553b4445bd232b13ebebd76f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:34 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39614
I20260808 04:49:34.227753 18751 tablet_service.cc:1514] Processing DeleteTablet for tablet b097c64c553b4445bd232b13ebebd76f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:34 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:47918
I20260808 04:49:34.228242 19298 tablet_replica.cc:330] T b097c64c553b4445bd232b13ebebd76f P 4548d6804d8a42a5a7e9e88f5267fcfe: stopping tablet replica
I20260808 04:49:34.228281 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 18666
I20260808 04:49:34.232760 19296 tablet_replica.cc:330] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: stopping tablet replica
I20260808 04:49:34.233024 19297 tablet_replica.cc:330] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: stopping tablet replica
I20260808 04:49:34.233379 19297 raft_consensus.cc:2229] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:49:34.233744 19296 raft_consensus.cc:2229] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:34.234170 19297 raft_consensus.cc:2258] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:34.234546 19296 raft_consensus.cc:2258] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:34.237273 19296 ts_tablet_manager.cc:1905] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:34.240142 19296 ts_tablet_manager.cc:1918] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:34.241202 19297 ts_tablet_manager.cc:1905] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:34.243809 19297 ts_tablet_manager.cc:1918] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:34.243976 19297 log.cc:1196] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/b097c64c553b4445bd232b13ebebd76f
I20260808 04:49:34.244345 19297 ts_tablet_manager.cc:1939] T b097c64c553b4445bd232b13ebebd76f P 2ee9c42c2c1d473097d659dea48424c4: Deleting consensus metadata
I20260808 04:49:34.244488 19296 log.cc:1196] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/b097c64c553b4445bd232b13ebebd76f
I20260808 04:49:34.244923 19296 ts_tablet_manager.cc:1939] T b097c64c553b4445bd232b13ebebd76f P ef061c35c883470686984d0df078eecd: Deleting consensus metadata
I20260808 04:49:34.246204 18559 catalog_manager.cc:4841] TS 2ee9c42c2c1d473097d659dea48424c4 (127.22.2.130:41409): tablet b097c64c553b4445bd232b13ebebd76f (table test-table [id=1f219438aaf24e64bcbdf200570adc6d]) successfully deleted
I20260808 04:49:34.247709 18555 catalog_manager.cc:4841] TS ef061c35c883470686984d0df078eecd (127.22.2.131:40819): tablet b097c64c553b4445bd232b13ebebd76f (table test-table [id=1f219438aaf24e64bcbdf200570adc6d]) successfully deleted
W20260808 04:49:34.250730 18557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:39869 (error 108)
I20260808 04:49:34.251752 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.251447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:39869 (local address 127.0.0.1:47930)
0808 04:49:34.251588 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:34.251661 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:39869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 04:49:34.251824 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 18865
W20260808 04:49:34.261431 18557 catalog_manager.cc:4584] TS 4548d6804d8a42a5a7e9e88f5267fcfe (127.22.2.129:39869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b097c64c553b4445bd232b13ebebd76f: Network error: Client connection negotiation failed: client connection to 127.22.2.129:39869: connect: Connection refused (error 111)
I20260808 04:49:34.267611 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 19056
I20260808 04:49:34.279235 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 18531
2026-08-08T04:49:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2614 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T04:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:34Z Disabled control of system clock
I20260808 04:49:34.316268 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37791
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:38529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:37791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 04:49:34.442140 19304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:34.442570 19304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:34.442646 19304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:34.449434 19304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:34.449512 19304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:34.449541 19304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:34.449579 19304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:34.449621 19304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:34.455168 19304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:37791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:34.456548 19304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:34.458331 19304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:34.468824 19313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:34.468958 19319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:34.468807 19311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:34.469286 19304 server_base.cc:1034] running on GCE node
I20260808 04:49:34.469884 19304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:34.471062 19304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:34.472251 19304 hybrid_clock.cc:648] HybridClock initialized: now 1786164574472224 us; error 40 us; skew 500 ppm
I20260808 04:49:34.474123 19304 webserver.cc:460] Webserver started at http://127.22.2.190:35839/ using document root <none> and password file <none>
I20260808 04:49:34.474762 19304 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:34.474860 19304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:34.475078 19304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:34.477188 19304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "1e3c820e4552415b8f3f42cb504e8124"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "5055ae03826a73105afe872eab064570"
server_key_iv: "fdfc4f7b7dd205e10da2e275ed5a8c80"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.477768 19304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "1e3c820e4552415b8f3f42cb504e8124"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "5055ae03826a73105afe872eab064570"
server_key_iv: "fdfc4f7b7dd205e10da2e275ed5a8c80"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.481534 19304 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:49:34.484045 19325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:34.485352 19304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:49:34.485500 19304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "1e3c820e4552415b8f3f42cb504e8124"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "5055ae03826a73105afe872eab064570"
server_key_iv: "fdfc4f7b7dd205e10da2e275ed5a8c80"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.485612 19304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:34.520711 19304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:34.521456 19304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:34.521672 19304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:34.529007 19304 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:37791
I20260808 04:49:34.529022 19392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:37791 every 8 connection(s)
I20260808 04:49:34.529976 19304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:34.532317 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 19304
I20260808 04:49:34.532481 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:34.532833 22538 external_mini_cluster.cc:1428] Setting key 7a7f8429a840593a70d4ad04812c6f5a
I20260808 04:49:34.533421 19393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:34.539784 19393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124: Bootstrap starting.
I20260808 04:49:34.542282 19393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:34.542933 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.534345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52338 (local address 127.22.2.190:37791)
0808 04:49:34.535053 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.535081 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.535121 (+ 40us) server_negotiation.cc:407] Connection header received
0808 04:49:34.535809 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.535826 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.536284 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.536681 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:34.537140 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.538132 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.539073 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.539560 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.539753 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.540944 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:34.540949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:34.540963 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:34.541400 (+ 437us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:34.541463 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:34.541484 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:34.541608 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.541769 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.542059 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":158,"threads_started":1}
I20260808 04:49:34.543081 19393 log.cc:824] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:34.545104 19393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124: No bootstrap required, opened a new log
I20260808 04:49:34.548478 19393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3c820e4552415b8f3f42cb504e8124" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37791 } }
I20260808 04:49:34.548740 19393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:34.548781 19393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3c820e4552415b8f3f42cb504e8124, State: Initialized, Role: FOLLOWER
I20260808 04:49:34.549177 19393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3c820e4552415b8f3f42cb504e8124" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37791 } }
I20260808 04:49:34.549296 19393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:34.549335 19393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:34.549398 19393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:34.550347 19393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3c820e4552415b8f3f42cb504e8124" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37791 } }
I20260808 04:49:34.550654 19393 leader_election.cc:302] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3c820e4552415b8f3f42cb504e8124; no voters:
I20260808 04:49:34.550918 19393 leader_election.cc:288] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:34.551086 19398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:34.551347 19398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [term 1 LEADER]: Becoming Leader. State: Replica: 1e3c820e4552415b8f3f42cb504e8124, State: Running, Role: LEADER
I20260808 04:49:34.551671 19398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3c820e4552415b8f3f42cb504e8124" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37791 } }
I20260808 04:49:34.552081 19393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:34.553398 19401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e3c820e4552415b8f3f42cb504e8124. Latest consensus state: current_term: 1 leader_uuid: "1e3c820e4552415b8f3f42cb504e8124" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3c820e4552415b8f3f42cb504e8124" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37791 } } }
I20260808 04:49:34.553452 19400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e3c820e4552415b8f3f42cb504e8124" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3c820e4552415b8f3f42cb504e8124" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37791 } } }
I20260808 04:49:34.553555 19401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:34.553555 19400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:34.554013 19412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:34.557291 19412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:34.562943 19412 catalog_manager.cc:1347] Generated new cluster ID: 8c48e571f561495daa5f18455bfefd43
I20260808 04:49:34.563020 19412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:34.576205 19412 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:34.577179 19412 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:34.584983 19412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124: Generated new TSK 0
I20260808 04:49:34.585633 19412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:34.598431 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37791
--builtin_ntp_servers=127.22.2.148:38529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 04:49:34.721079 19431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:34.721329 19431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:34.721366 19431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:34.725404 19431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:34.725450 19431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:34.725528 19431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 04:49:34.730187 19431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:34.731202 19431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:34.732602 19431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:34.742372 19442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:34.742499 19431 server_base.cc:1034] running on GCE node
W20260808 04:49:34.742419 19437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:34.742333 19438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:34.743106 19431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:34.743732 19431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:34.744922 19431 hybrid_clock.cc:648] HybridClock initialized: now 1786164574744906 us; error 28 us; skew 500 ppm
I20260808 04:49:34.746702 19431 webserver.cc:460] Webserver started at http://127.22.2.129:38885/ using document root <none> and password file <none>
I20260808 04:49:34.747288 19431 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:34.747381 19431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:34.747618 19431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:34.749712 19431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/data/instance:
uuid: "ea67ff917eed49d1a25e06c56eac31c1"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "29225a2be8123cb9ebabff2b51eade28"
server_key_iv: "c0cfb851e732f4efc2ae2143d47bf8ae"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.750283 19431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "ea67ff917eed49d1a25e06c56eac31c1"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "29225a2be8123cb9ebabff2b51eade28"
server_key_iv: "c0cfb851e732f4efc2ae2143d47bf8ae"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.754160 19431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:49:34.756901 19452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:34.758234 19431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:49:34.758386 19431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/wal
uuid: "ea67ff917eed49d1a25e06c56eac31c1"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "29225a2be8123cb9ebabff2b51eade28"
server_key_iv: "c0cfb851e732f4efc2ae2143d47bf8ae"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.758517 19431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:34.783964 19431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:34.784803 19431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:34.785019 19431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:34.785666 19431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:34.786698 19431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:34.786777 19431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:34.786860 19431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:34.786911 19431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:34.797115 19431 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:34845
I20260808 04:49:34.797147 19617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:34845 every 8 connection(s)
I20260808 04:49:34.798092 19431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 04:49:34.806424 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 19431
I20260808 04:49:34.806564 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-0/wal/instance
I20260808 04:49:34.806924 22538 external_mini_cluster.cc:1428] Setting key 03087001c2381693c181d5017bc0f402
I20260808 04:49:34.808558 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.800093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:53187 (local address 127.22.2.190:37791)
0808 04:49:34.800252 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.800256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.801528 (+ 1272us) server_negotiation.cc:407] Connection header received
0808 04:49:34.803175 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.803178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.803231 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.803310 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:34.804235 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.804837 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.806158 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.806518 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.806645 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.807963 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:34.807966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:34.807968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:34.807979 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:34.807998 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:34.808006 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:34.808067 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.808274 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.808368 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 04:49:34.809213 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37791
--builtin_ntp_servers=127.22.2.148:38529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 04:49:34.809806 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.800469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37791 (local address 127.22.2.129:53187)
0808 04:49:34.801338 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:34.801398 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:34.802924 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:34.803488 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:34.803494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:34.803590 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:34.804096 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.804110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.804976 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.804980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.806008 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.806019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.806744 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.806748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.806905 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.807628 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:34.807629 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:34.807671 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:34.807778 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:34.808098 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:34.808115 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:34.808121 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 04:49:34.808263 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 04:49:34.808748 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":112,"threads_started":1}
I20260808 04:49:34.811293 19618 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37791
I20260808 04:49:34.811560 19618 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:34.812187 19618 heartbeater.cc:511] Master 127.22.2.190:37791 requested a full tablet report, sending...
I20260808 04:49:34.813835 19350 ts_manager.cc:194] Registered new tserver with Master: ea67ff917eed49d1a25e06c56eac31c1 (127.22.2.129:34845)
I20260808 04:49:34.815084 19350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.129:53187
W20260808 04:49:34.925967 19621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:34.926218 19621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:34.926255 19621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:34.930927 19621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:34.930982 19621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:34.931069 19621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 04:49:34.935719 19621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:34.936865 19621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:34.938398 19621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:34.948674 19629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:34.948657 19627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:34.948779 19631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:34.949100 19621 server_base.cc:1034] running on GCE node
I20260808 04:49:34.949530 19621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:34.950166 19621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:34.951361 19621 hybrid_clock.cc:648] HybridClock initialized: now 1786164574951347 us; error 26 us; skew 500 ppm
I20260808 04:49:34.953410 19621 webserver.cc:460] Webserver started at http://127.22.2.130:44927/ using document root <none> and password file <none>
I20260808 04:49:34.954041 19621 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:34.954136 19621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:34.954453 19621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:34.957003 19621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/data/instance:
uuid: "3313d706c4d94877bc6a04a2f11f229c"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "466806578c879f165dac585ae1169f8f"
server_key_iv: "56875023f4eaa83619e9319a7df0929d"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.957576 19621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "3313d706c4d94877bc6a04a2f11f229c"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "466806578c879f165dac585ae1169f8f"
server_key_iv: "56875023f4eaa83619e9319a7df0929d"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.961329 19621 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:49:34.963860 19638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:34.965170 19621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:49:34.965310 19621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/wal
uuid: "3313d706c4d94877bc6a04a2f11f229c"
format_stamp: "Formatted at 2026-08-08 04:49:34 on dist-test-slave-04bb"
server_key: "466806578c879f165dac585ae1169f8f"
server_key_iv: "56875023f4eaa83619e9319a7df0929d"
server_key_version: "encryptionkey@0"
I20260808 04:49:34.965420 19621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:34.975237 19621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:34.975855 19621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:34.976043 19621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:34.976698 19621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:34.977644 19621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:34.977720 19621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:34.977802 19621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:34.977855 19621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:34.987063 19621 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35769
I20260808 04:49:34.987092 19814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35769 every 8 connection(s)
I20260808 04:49:34.987993 19621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 04:49:34.997200 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 19621
I20260808 04:49:34.997382 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/wal/instance
I20260808 04:49:34.997684 22538 external_mini_cluster.cc:1428] Setting key 6c422c7da6adb53c77867270cb3cb5a5
I20260808 04:49:34.998262 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.990028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:60689 (local address 127.22.2.190:37791)
0808 04:49:34.990291 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:34.990297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:34.991515 (+ 1218us) server_negotiation.cc:407] Connection header received
0808 04:49:34.993174 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:34.993178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:34.993227 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:34.993300 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:34.994288 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.994799 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.996079 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.996296 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.996382 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.997781 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:34.997785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:34.997788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:34.997803 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:34.997825 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:34.997837 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:34.997921 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:34.998077 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 04:49:34.998140 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260808 04:49:34.999305 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:34.990563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37791 (local address 127.22.2.130:60689)
0808 04:49:34.991349 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:34.991397 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:34.992926 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:34.993479 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:34.993487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:34.993575 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:34.994146 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.994161 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.994911 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.994914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.995950 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:34.995962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:34.996475 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:34.996478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:34.996770 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:34.997482 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:34.997484 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:34.997539 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:34.997600 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:34.997931 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:34.997938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:34.997943 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:49:34.998042 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 04:49:34.998462 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":135,"threads_started":1}
I20260808 04:49:34.999786 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37791
--builtin_ntp_servers=127.22.2.148:38529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 04:49:35.000456 19815 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37791
I20260808 04:49:35.000697 19815 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:35.001206 19815 heartbeater.cc:511] Master 127.22.2.190:37791 requested a full tablet report, sending...
I20260808 04:49:35.002269 19350 ts_manager.cc:194] Registered new tserver with Master: 3313d706c4d94877bc6a04a2f11f229c (127.22.2.130:35769)
I20260808 04:49:35.003082 19350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.130:60689
W20260808 04:49:35.115322 19822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:35.115561 19822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:35.115598 19822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:35.119586 19822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:35.119638 19822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:35.119689 19822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 04:49:35.124264 19822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:35.125264 19822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:35.126605 19822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:35.136116 19829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:35.136106 19832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:35.136216 19822 server_base.cc:1034] running on GCE node
W20260808 04:49:35.136118 19836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:35.136818 19822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:35.137481 19822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:35.138691 19822 hybrid_clock.cc:648] HybridClock initialized: now 1786164575138654 us; error 58 us; skew 500 ppm
I20260808 04:49:35.140592 19822 webserver.cc:460] Webserver started at http://127.22.2.131:42923/ using document root <none> and password file <none>
I20260808 04:49:35.141192 19822 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:35.141291 19822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:35.141542 19822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:35.143612 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/data/instance:
uuid: "acad60ec14ca4bc8855653dfa71b5656"
format_stamp: "Formatted at 2026-08-08 04:49:35 on dist-test-slave-04bb"
server_key: "56fdddbc80f06e16d57ae56e59fe2ae3"
server_key_iv: "1127e97ef47e23441b0495d72c15f8e4"
server_key_version: "encryptionkey@0"
I20260808 04:49:35.144222 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "acad60ec14ca4bc8855653dfa71b5656"
format_stamp: "Formatted at 2026-08-08 04:49:35 on dist-test-slave-04bb"
server_key: "56fdddbc80f06e16d57ae56e59fe2ae3"
server_key_iv: "1127e97ef47e23441b0495d72c15f8e4"
server_key_version: "encryptionkey@0"
I20260808 04:49:35.147945 19822 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:49:35.150444 19845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:35.151592 19822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:35.151741 19822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/wal
uuid: "acad60ec14ca4bc8855653dfa71b5656"
format_stamp: "Formatted at 2026-08-08 04:49:35 on dist-test-slave-04bb"
server_key: "56fdddbc80f06e16d57ae56e59fe2ae3"
server_key_iv: "1127e97ef47e23441b0495d72c15f8e4"
server_key_version: "encryptionkey@0"
I20260808 04:49:35.151855 19822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:35.180302 19822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:35.181044 19822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:35.181254 19822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:35.181864 19822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:49:35.182833 19822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:49:35.182919 19822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:35.183014 19822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:49:35.183068 19822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:35.192415 19822 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:44285
I20260808 04:49:35.192467 20024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:44285 every 8 connection(s)
I20260808 04:49:35.193398 19822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 04:49:35.197381 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 19822
I20260808 04:49:35.197505 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/wal/instance
I20260808 04:49:35.197784 22538 external_mini_cluster.cc:1428] Setting key 7cd7f796aada443cff50cf4473d400c9
I20260808 04:49:35.203189 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.195245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:51977 (local address 127.22.2.190:37791)
0808 04:49:35.195392 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:35.195396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:35.196458 (+ 1062us) server_negotiation.cc:407] Connection header received
0808 04:49:35.198043 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:35.198047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:35.198108 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:35.198221 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:35.199435 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.199975 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.201212 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.201441 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.201556 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.202669 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:35.202673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:35.202675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:35.202689 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:35.202715 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:35.202723 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:35.202808 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:35.202968 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 04:49:35.203030 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 04:49:35.204033 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.195657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37791 (local address 127.22.2.131:51977)
0808 04:49:35.196305 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:35.196346 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:35.197797 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:35.198399 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:35.198423 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:35.198528 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:49:35.199268 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.199288 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.200107 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.200110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.201082 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.201092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.201597 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.201601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.201767 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.202411 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:49:35.202413 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:49:35.202453 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:49:35.202514 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:49:35.202810 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:49:35.202822 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:49:35.202831 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 04:49:35.202965 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 04:49:35.203249 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":115,"threads_started":1}
I20260808 04:49:35.205190 20025 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37791
I20260808 04:49:35.205452 20025 heartbeater.cc:464] Registering TS with master...
I20260808 04:49:35.205977 20025 heartbeater.cc:511] Master 127.22.2.190:37791 requested a full tablet report, sending...
I20260808 04:49:35.206998 19350 ts_manager.cc:194] Registered new tserver with Master: acad60ec14ca4bc8855653dfa71b5656 (127.22.2.131:44285)
I20260808 04:49:35.207553 19350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.131:51977
I20260808 04:49:35.211843 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:49:35.217424 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.213809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52346 (local address 127.22.2.190:37791)
0808 04:49:35.213945 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:35.213948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:35.214159 (+ 211us) server_negotiation.cc:407] Connection header received
0808 04:49:35.214270 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:35.214272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:35.214319 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:35.214386 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:35.214654 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.215126 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.215979 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.216181 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.216254 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.217070 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:35.217073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:35.217075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:35.217086 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:35.217104 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:35.217111 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:35.217169 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:35.217269 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 04:49:35.217327 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 04:49:35.220093 19350 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:49:35.223057 19350 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:49:35.237841 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.231512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:34845 (local address 127.0.0.1:41660)
0808 04:49:35.232084 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:35.232134 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:35.232273 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:35.233211 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:35.233218 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:35.233244 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:35.233516 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.233527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.234991 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.234994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.236329 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.236344 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.237292 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.237296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.237403 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.237440 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 04:49:35.237549 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 04:49:35.237680 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":141,"spinlock_wait_cycles":4480,"thread_start_us":89,"threads_started":1}
I20260808 04:49:35.237843 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.232194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.22.2.129:34845)
0808 04:49:35.232705 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:35.232716 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:35.232739 (+ 23us) server_negotiation.cc:407] Connection header received
0808 04:49:35.232819 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:35.232824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:35.233020 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:35.233409 (+ 389us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:35.233670 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.234873 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.236467 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.237118 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.237293 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.237497 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:35.237588 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 04:49:35.237683 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":144,"threads_started":1}
I20260808 04:49:35.238229 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.231385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35769 (local address 127.0.0.1:44362)
0808 04:49:35.232086 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:35.232136 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:35.232272 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:35.233313 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:35.233315 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:35.233330 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:35.233552 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.233561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.235037 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.235040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.236366 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.236376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.237667 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.237671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.237828 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.237919 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 04:49:35.237992 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:49:35.238079 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":137,"spinlock_wait_cycles":4992,"thread_start_us":76,"threads_started":1}
I20260808 04:49:35.238329 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.232539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.22.2.131:44285)
0808 04:49:35.233203 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:35.233216 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:35.233235 (+ 19us) server_negotiation.cc:407] Connection header received
0808 04:49:35.233303 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:35.233307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:35.233468 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:35.233625 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:35.233860 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.234599 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.236372 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.237308 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.237422 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.237595 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:35.238074 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 04:49:35.238166 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":94,"threads_started":1}
I20260808 04:49:35.238528 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.231828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:44285 (local address 127.0.0.1:60472)
0808 04:49:35.232287 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:35.232309 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:35.232385 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:35.233614 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:35.233617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:35.233636 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:35.233736 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.233743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.234721 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.234725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.236187 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.236200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.237667 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.237671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.237851 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.237924 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 04:49:35.237992 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 04:49:35.238417 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":440,"thread_start_us":88,"threads_started":1}
I20260808 04:49:35.238960 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.231528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.22.2.130:35769)
0808 04:49:35.232636 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:35.232650 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:35.232677 (+ 27us) server_negotiation.cc:407] Connection header received
0808 04:49:35.232775 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:35.232781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:35.232971 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:35.233147 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:35.233678 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.234873 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.236519 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.237333 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.237422 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.237595 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:35.238699 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0808 04:49:35.238800 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":156,"threads_started":1}
I20260808 04:49:35.241302 19718 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba2dcf8eb34848a834638c3a4330e4 (DEFAULT_TABLE table=test-table [id=5d0b18269a234f189b8079edacc6ad73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:35.241240 19525 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba2dcf8eb34848a834638c3a4330e4 (DEFAULT_TABLE table=test-table [id=5d0b18269a234f189b8079edacc6ad73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:35.241238 19926 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba2dcf8eb34848a834638c3a4330e4 (DEFAULT_TABLE table=test-table [id=5d0b18269a234f189b8079edacc6ad73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:49:35.242250 19718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba2dcf8eb34848a834638c3a4330e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:35.242273 19926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba2dcf8eb34848a834638c3a4330e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:35.242273 19525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba2dcf8eb34848a834638c3a4330e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:35.248482 20055 tablet_bootstrap.cc:492] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: Bootstrap starting.
I20260808 04:49:35.250077 20056 tablet_bootstrap.cc:492] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1: Bootstrap starting.
I20260808 04:49:35.250077 20057 tablet_bootstrap.cc:492] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: Bootstrap starting.
I20260808 04:49:35.250686 20055 tablet_bootstrap.cc:654] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:35.251386 20055 log.cc:824] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:35.252167 20057 tablet_bootstrap.cc:654] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:35.252177 20056 tablet_bootstrap.cc:654] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:35.253011 20056 log.cc:824] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:35.253011 20057 log.cc:824] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:35.253567 20055 tablet_bootstrap.cc:492] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: No bootstrap required, opened a new log
I20260808 04:49:35.253772 20055 ts_tablet_manager.cc:1397] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 04:49:35.254922 20057 tablet_bootstrap.cc:492] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: No bootstrap required, opened a new log
I20260808 04:49:35.255163 20057 ts_tablet_manager.cc:1397] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 04:49:35.255522 20056 tablet_bootstrap.cc:492] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1: No bootstrap required, opened a new log
I20260808 04:49:35.255714 20056 ts_tablet_manager.cc:1397] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:49:35.257436 20055 raft_consensus.cc:348] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.257674 20055 raft_consensus.cc:374] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:35.257738 20055 raft_consensus.cc:729] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3313d706c4d94877bc6a04a2f11f229c, State: Initialized, Role: FOLLOWER
I20260808 04:49:35.258198 20055 consensus_queue.cc:260] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.258875 19815 heartbeater.cc:503] Master 127.22.2.190:37791 was elected leader, sending a full tablet report...
I20260808 04:49:35.259191 20055 ts_tablet_manager.cc:1428] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 04:49:35.259366 20057 raft_consensus.cc:348] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.259610 20057 raft_consensus.cc:374] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:35.259681 20057 raft_consensus.cc:729] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acad60ec14ca4bc8855653dfa71b5656, State: Initialized, Role: FOLLOWER
I20260808 04:49:35.260133 20057 consensus_queue.cc:260] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.261091 20057 ts_tablet_manager.cc:1428] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 04:49:35.261053 20056 raft_consensus.cc:348] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.261317 20056 raft_consensus.cc:374] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:35.261374 20056 raft_consensus.cc:729] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea67ff917eed49d1a25e06c56eac31c1, State: Initialized, Role: FOLLOWER
I20260808 04:49:35.261437 20025 heartbeater.cc:503] Master 127.22.2.190:37791 was elected leader, sending a full tablet report...
I20260808 04:49:35.261778 20056 consensus_queue.cc:260] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.262689 20056 ts_tablet_manager.cc:1428] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 04:49:35.263715 19618 heartbeater.cc:503] Master 127.22.2.190:37791 was elected leader, sending a full tablet report...
W20260808 04:49:35.299461 19619 tablet.cc:2369] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:49:35.351814 20064 raft_consensus.cc:482] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:49:35.352056 20064 raft_consensus.cc:504] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.353515 20064 leader_election.cc:288] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea67ff917eed49d1a25e06c56eac31c1 (127.22.2.129:34845), acad60ec14ca4bc8855653dfa71b5656 (127.22.2.131:44285)
I20260808 04:49:35.357466 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.353964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:40989 (local address 127.22.2.129:34845)
0808 04:49:35.354107 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:35.354112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:35.354146 (+ 34us) server_negotiation.cc:407] Connection header received
0808 04:49:35.354194 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:35.354196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:35.354238 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:35.354330 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:35.354641 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.355274 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.356434 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.356884 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.357013 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.357068 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:35.357283 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 04:49:35.357355 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 04:49:35.357470 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.353705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:34845 (local address 127.22.2.130:40989)
0808 04:49:35.353911 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:35.353928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:35.354019 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:35.354345 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:35.354354 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:35.354377 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:35.354505 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.354511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.355386 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.355391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.356307 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.356317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.357013 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.357017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.357123 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.357173 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 04:49:35.357263 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:49:35.357332 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":25,"spinlock_wait_cycles":10624}
I20260808 04:49:35.357579 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.353956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:35097 (local address 127.22.2.131:44285)
0808 04:49:35.354121 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:35.354126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:35.354318 (+ 192us) server_negotiation.cc:407] Connection header received
0808 04:49:35.354503 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:35.354505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:35.354545 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:35.354628 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:49:35.354915 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.355415 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.356578 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.356991 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.357128 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.357181 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:35.357383 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 04:49:35.357450 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 04:49:35.357632 20067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.353783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:44285 (local address 127.22.2.130:35097)
0808 04:49:35.354228 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:35.354239 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:49:35.354402 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:49:35.354639 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:49:35.354642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:49:35.354677 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:49:35.354801 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.354807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.355525 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.355530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.356436 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:49:35.356446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.357129 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.357133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:49:35.357258 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.357275 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 04:49:35.357365 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:49:35.357429 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"spinlock_wait_cycles":12032,"thread_start_us":129,"threads_started":1}
I20260808 04:49:35.358179 19553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba2dcf8eb34848a834638c3a4330e4" candidate_uuid: "3313d706c4d94877bc6a04a2f11f229c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea67ff917eed49d1a25e06c56eac31c1" is_pre_election: true
I20260808 04:49:35.358222 19960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba2dcf8eb34848a834638c3a4330e4" candidate_uuid: "3313d706c4d94877bc6a04a2f11f229c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acad60ec14ca4bc8855653dfa71b5656" is_pre_election: true
I20260808 04:49:35.358480 19553 raft_consensus.cc:2454] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3313d706c4d94877bc6a04a2f11f229c in term 0.
I20260808 04:49:35.358584 19960 raft_consensus.cc:2454] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3313d706c4d94877bc6a04a2f11f229c in term 0.
I20260808 04:49:35.359170 19643 leader_election.cc:302] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3313d706c4d94877bc6a04a2f11f229c, acad60ec14ca4bc8855653dfa71b5656, ea67ff917eed49d1a25e06c56eac31c1; no voters:
I20260808 04:49:35.359819 20064 raft_consensus.cc:2781] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:49:35.359973 20064 raft_consensus.cc:482] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:49:35.360049 20064 raft_consensus.cc:3037] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:35.361229 20064 raft_consensus.cc:504] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.361613 20064 leader_election.cc:288] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [CANDIDATE]: Term 1 election: Requested vote from peers ea67ff917eed49d1a25e06c56eac31c1 (127.22.2.129:34845), acad60ec14ca4bc8855653dfa71b5656 (127.22.2.131:44285)
I20260808 04:49:35.361963 19553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba2dcf8eb34848a834638c3a4330e4" candidate_uuid: "3313d706c4d94877bc6a04a2f11f229c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea67ff917eed49d1a25e06c56eac31c1"
I20260808 04:49:35.362130 19553 raft_consensus.cc:3037] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:35.362994 19960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba2dcf8eb34848a834638c3a4330e4" candidate_uuid: "3313d706c4d94877bc6a04a2f11f229c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acad60ec14ca4bc8855653dfa71b5656"
I20260808 04:49:35.363132 19960 raft_consensus.cc:3037] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:35.363759 19553 raft_consensus.cc:2454] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3313d706c4d94877bc6a04a2f11f229c in term 1.
I20260808 04:49:35.364739 19960 raft_consensus.cc:2454] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3313d706c4d94877bc6a04a2f11f229c in term 1.
I20260808 04:49:35.365168 19646 leader_election.cc:302] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3313d706c4d94877bc6a04a2f11f229c, acad60ec14ca4bc8855653dfa71b5656; no voters:
I20260808 04:49:35.365434 20064 raft_consensus.cc:2781] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:35.365657 20064 raft_consensus.cc:686] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 1 LEADER]: Becoming Leader. State: Replica: 3313d706c4d94877bc6a04a2f11f229c, State: Running, Role: LEADER
I20260808 04:49:35.366039 20064 consensus_queue.cc:237] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } }
I20260808 04:49:35.370432 19349 catalog_manager.cc:5495] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3313d706c4d94877bc6a04a2f11f229c (127.22.2.130). New cstate: current_term: 1 leader_uuid: "3313d706c4d94877bc6a04a2f11f229c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3313d706c4d94877bc6a04a2f11f229c" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 } health_report { overall_health: UNKNOWN } } }
I20260808 04:49:35.388516 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.385349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51888 (local address 127.22.2.130:35769)
0808 04:49:35.385474 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:35.385477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:35.385491 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:49:35.385591 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:35.385594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:35.385648 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:35.385738 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:49:35.385919 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.386415 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.387203 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:35.387406 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:35.387483 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:35.387731 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:49:35.388129 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:49:35.388215 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:35.388335 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 04:49:35.388389 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 04:49:35.394472 19553 raft_consensus.cc:1261] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 1 FOLLOWER]: Refusing update from remote peer 3313d706c4d94877bc6a04a2f11f229c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:35.394472 19960 raft_consensus.cc:1261] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 1 FOLLOWER]: Refusing update from remote peer 3313d706c4d94877bc6a04a2f11f229c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:49:35.395179 20064 consensus_queue.cc:1035] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea67ff917eed49d1a25e06c56eac31c1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 34845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:35.395341 20070 consensus_queue.cc:1035] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [LEADER]: Connected to new peer: Peer: permanent_uuid: "acad60ec14ca4bc8855653dfa71b5656" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:49:35.405637 20080 mvcc.cc:205] Tried to move back new op lower bound from 7316130100808531968 to 7316130100702945280. Current Snapshot: MvccSnapshot[applied={T|T < 7316130100808531968}]
I20260808 04:49:35.407939 20078 mvcc.cc:205] Tried to move back new op lower bound from 7316130100808531968 to 7316130100702945280. Current Snapshot: MvccSnapshot[applied={T|T < 7316130100808531968 or (T in {7316130100808531968})}]
I20260808 04:49:35.410457 19350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:35.410588 19350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:49:35.413028 19350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e3c820e4552415b8f3f42cb504e8124: Sending DeleteTablet for 3 replicas of tablet a1ba2dcf8eb34848a834638c3a4330e4
I20260808 04:49:35.413856 19926 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba2dcf8eb34848a834638c3a4330e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:35 UTC) from {username='slave'} at 127.0.0.1:60472
I20260808 04:49:35.413862 19718 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba2dcf8eb34848a834638c3a4330e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:35 UTC) from {username='slave'} at 127.0.0.1:44362
I20260808 04:49:35.413872 19525 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba2dcf8eb34848a834638c3a4330e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:49:35 UTC) from {username='slave'} at 127.0.0.1:41660
I20260808 04:49:35.414377 20089 tablet_replica.cc:330] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1: stopping tablet replica
I20260808 04:49:35.414547 20090 tablet_replica.cc:330] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: stopping tablet replica
I20260808 04:49:35.414777 20089 raft_consensus.cc:2229] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:35.415082 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 19431
I20260808 04:49:35.415094 20089 raft_consensus.cc:2258] T a1ba2dcf8eb34848a834638c3a4330e4 P ea67ff917eed49d1a25e06c56eac31c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:35.415444 20088 tablet_replica.cc:330] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: stopping tablet replica
I20260808 04:49:35.420315 20088 raft_consensus.cc:2229] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:49:35.420644 20088 raft_consensus.cc:2258] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:35.422627 20088 ts_tablet_manager.cc:1905] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:35.425398 20088 ts_tablet_manager.cc:1918] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:35.425496 20088 log.cc:1196] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-2/wal/wals/a1ba2dcf8eb34848a834638c3a4330e4
I20260808 04:49:35.425812 20088 ts_tablet_manager.cc:1939] T a1ba2dcf8eb34848a834638c3a4330e4 P acad60ec14ca4bc8855653dfa71b5656: Deleting consensus metadata
I20260808 04:49:35.432848 20090 raft_consensus.cc:2229] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:49:35.433265 20090 raft_consensus.cc:2258] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:49:35.437353 19331 catalog_manager.cc:4841] TS acad60ec14ca4bc8855653dfa71b5656 (127.22.2.131:44285): tablet a1ba2dcf8eb34848a834638c3a4330e4 (table test-table [id=5d0b18269a234f189b8079edacc6ad73]) successfully deleted
I20260808 04:49:35.440388 20090 ts_tablet_manager.cc:1905] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:49:35.442698 20090 ts_tablet_manager.cc:1918] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:49:35.442813 20090 log.cc:1196] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: Deleting WAL directory at /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786164310373382-22538-0/minicluster-data/ts-1/wal/wals/a1ba2dcf8eb34848a834638c3a4330e4
I20260808 04:49:35.443105 20090 ts_tablet_manager.cc:1939] T a1ba2dcf8eb34848a834638c3a4330e4 P 3313d706c4d94877bc6a04a2f11f229c: Deleting consensus metadata
I20260808 04:49:35.444227 19327 catalog_manager.cc:4841] TS 3313d706c4d94877bc6a04a2f11f229c (127.22.2.130:35769): tablet a1ba2dcf8eb34848a834638c3a4330e4 (table test-table [id=5d0b18269a234f189b8079edacc6ad73]) successfully deleted
W20260808 04:49:35.446370 19328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:34845 (error 108)
I20260808 04:49:35.446564 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 19621
I20260808 04:49:35.450704 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:35.450207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:34845 (local address 127.0.0.1:42934)
0808 04:49:35.450475 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 04:49:35.450581 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260808 04:49:35.450934 19328 catalog_manager.cc:4584] TS ea67ff917eed49d1a25e06c56eac31c1 (127.22.2.129:34845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1ba2dcf8eb34848a834638c3a4330e4: Network error: Client connection negotiation failed: client connection to 127.22.2.129:34845: connect: Connection refused (error 111)
I20260808 04:49:35.459244 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 19822
I20260808 04:49:35.468081 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 19304
2026-08-08T04:49:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1184 ms)
[----------] 6 tests from EncryptionPolicyTest (11855 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:35 dist-test-slave-04bb krb5kdc[20096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:35 dist-test-slave-04bb krb5kdc[20096](info): set up 2 sockets
Aug 08 04:49:35 dist-test-slave-04bb krb5kdc[20096](info): commencing operation
krb5kdc: starting...
W20260808 04:49:36.720163 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:36 dist-test-slave-04bb krb5kdc[20096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:36.851142 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:36.966763 20111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:36.967010 20111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:36.967052 20111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:36.970943 20111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:36.970999 20111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:36.971017 20111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:36.971033 20111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:36.971050 20111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:36.976117 20111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:36.977209 20111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:36.978188 20111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:36.986902 20119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:36.986959 20122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:36.986991 20120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:36.987046 20111 server_base.cc:1034] running on GCE node
I20260808 04:49:36.987774 20111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:36.988936 20111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:36.990097 20111 hybrid_clock.cc:648] HybridClock initialized: now 1786164576990085 us; error 26 us; skew 500 ppm
Aug 08 04:49:36 dist-test-slave-04bb krb5kdc[20096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164576, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:36.992944 20111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:36.993865 20111 webserver.cc:460] Webserver started at http://127.0.0.1:45811/ using document root <none> and password file <none>
I20260808 04:49:36.994463 20111 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:36.994561 20111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:36.994812 20111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:36.996915 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "744f7432e2e046c2a67c73d3713c128c"
format_stamp: "Formatted at 2026-08-08 04:49:36 on dist-test-slave-04bb"
server_key: "5160452befa1ef78eeb5059c9dbeb7be"
server_key_iv: "d1f86b8d5205387129e4ca22534e93af"
server_key_version: "encryptionkey@0"
I20260808 04:49:36.997537 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "744f7432e2e046c2a67c73d3713c128c"
format_stamp: "Formatted at 2026-08-08 04:49:36 on dist-test-slave-04bb"
server_key: "5160452befa1ef78eeb5059c9dbeb7be"
server_key_iv: "d1f86b8d5205387129e4ca22534e93af"
server_key_version: "encryptionkey@0"
I20260808 04:49:37.001441 20111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 04:49:37.003928 20133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:37.005137 20111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:49:37.005319 20111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "744f7432e2e046c2a67c73d3713c128c"
format_stamp: "Formatted at 2026-08-08 04:49:36 on dist-test-slave-04bb"
server_key: "5160452befa1ef78eeb5059c9dbeb7be"
server_key_iv: "d1f86b8d5205387129e4ca22534e93af"
server_key_version: "encryptionkey@0"
I20260808 04:49:37.005429 20111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:37.028137 20111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:37.031404 20111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:37.031615 20111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:37.039387 20111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41807
I20260808 04:49:37.039431 20210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41807 every 8 connection(s)
I20260808 04:49:37.040423 20111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:37.043359 20211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:37.048043 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 20111
I20260808 04:49:37.048199 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:37.048491 22538 external_mini_cluster.cc:1428] Setting key 7b4a6f01c58bc552c49f2fb6b7949d94
I20260808 04:49:37.049175 20211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c: Bootstrap starting.
I20260808 04:49:37.051514 20211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:37.052516 20211 log.cc:824] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:37.054688 20211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c: No bootstrap required, opened a new log
Aug 08 04:49:37 dist-test-slave-04bb krb5kdc[20096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 04:49:37.059127 20211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f7432e2e046c2a67c73d3713c128c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41807 } }
I20260808 04:49:37.059399 20211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:37.059504 20211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744f7432e2e046c2a67c73d3713c128c, State: Initialized, Role: FOLLOWER
I20260808 04:49:37.059994 20211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f7432e2e046c2a67c73d3713c128c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41807 } }
I20260808 04:49:37.060168 20211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:37.060263 20211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:37.060374 20211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:37.061254 20211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f7432e2e046c2a67c73d3713c128c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41807 } }
I20260808 04:49:37.061602 20211 leader_election.cc:302] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 744f7432e2e046c2a67c73d3713c128c; no voters:
I20260808 04:49:37.061990 20211 leader_election.cc:288] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:37.062052 20219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:37.062270 20219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [term 1 LEADER]: Becoming Leader. State: Replica: 744f7432e2e046c2a67c73d3713c128c, State: Running, Role: LEADER
I20260808 04:49:37.062574 20219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f7432e2e046c2a67c73d3713c128c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41807 } }
I20260808 04:49:37.063292 20211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:37.063771 20219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "744f7432e2e046c2a67c73d3713c128c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f7432e2e046c2a67c73d3713c128c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41807 } } }
I20260808 04:49:37.063896 20219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:37.064186 20221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 744f7432e2e046c2a67c73d3713c128c. Latest consensus state: current_term: 1 leader_uuid: "744f7432e2e046c2a67c73d3713c128c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f7432e2e046c2a67c73d3713c128c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41807 } } }
I20260808 04:49:37.064273 20221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:37.064662 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:37.050052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37361 (local address 127.0.0.1:41807)
0808 04:49:37.050478 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:37.050495 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:37.050533 (+ 38us) server_negotiation.cc:407] Connection header received
0808 04:49:37.051136 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:37.051162 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:37.051497 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:37.051825 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:37.052828 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:37.053986 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:37.055136 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:37.055591 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:37.055721 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:37.058577 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:37.058582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:37.058596 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:37.058621 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:37.060694 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:37.061209 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:37.061221 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:37.061225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:37.061335 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:37.061776 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:37.061779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:37.061781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:37.062233 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:37.063092 (+ 859us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:37.063238 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 04:49:37.063458 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":146,"threads_started":1}
I20260808 04:49:37.068024 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 04:49:37.068495 20244 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:49:37.068615 20244 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:49:37.068681 20224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:37.069537 20224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:37.075450 20224 catalog_manager.cc:1347] Generated new cluster ID: 7b4dd3cb41ba4972ad0a7a6ef6d0fd94
I20260808 04:49:37.075522 20224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:37.080108 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:37.070138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60393 (local address 127.0.0.1:41807)
0808 04:49:37.070256 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:37.070261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:37.071796 (+ 1535us) server_negotiation.cc:407] Connection header received
0808 04:49:37.071909 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:37.071912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:37.071958 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:37.072040 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:37.072875 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:37.073548 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:37.074455 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:37.074723 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:37.074834 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:37.076622 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:37.076626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:37.076628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:37.076662 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:37.078264 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:37.078794 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:37.078797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:37.078799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:37.078871 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:37.079196 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:37.079199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:37.079201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:37.079396 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:37.079502 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:37.079880 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 04:49:37.079969 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:49:37.098121 20224 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:37.099345 20224 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:37.106117 20224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 744f7432e2e046c2a67c73d3713c128c: Generated new TSK 0
I20260808 04:49:37.106896 20224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:37.114990 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 20111
W20260808 04:49:37.128070 20093 connection.cc:535] client connection to 127.0.0.1:41807 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41807): Connection reset by peer (error 104)
2026-08-08T04:49:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1650 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:37 dist-test-slave-04bb krb5kdc[20259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:37 dist-test-slave-04bb krb5kdc[20259](info): set up 2 sockets
Aug 08 04:49:37 dist-test-slave-04bb krb5kdc[20259](info): commencing operation
krb5kdc: starting...
W20260808 04:49:38.358983 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:38 dist-test-slave-04bb krb5kdc[20259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:38.489553 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:38.604352 20274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:38.604607 20274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:38.604650 20274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:38.608616 20274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:38.608669 20274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:38.608688 20274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:38.608704 20274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:38.608719 20274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:38.613835 20274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:38.614830 20274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:38.615851 20274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:38.624792 20284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:38.624836 20282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:38.624837 20281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:38.625162 20274 server_base.cc:1034] running on GCE node
I20260808 04:49:38.625756 20274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:38.626785 20274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:38.628013 20274 hybrid_clock.cc:648] HybridClock initialized: now 1786164578627957 us; error 68 us; skew 500 ppm
Aug 08 04:49:38 dist-test-slave-04bb krb5kdc[20259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164578, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:38.631307 20274 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:38.632372 20274 webserver.cc:460] Webserver started at http://127.0.0.1:44301/ using document root <none> and password file <none>
I20260808 04:49:38.633219 20274 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:38.633325 20274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:38.633606 20274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:38.636504 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "560e5076c7d3441aa835c4f6b38cb01b"
format_stamp: "Formatted at 2026-08-08 04:49:38 on dist-test-slave-04bb"
server_key: "3b3806cb91fd74cdbb3c213ab5733baf"
server_key_iv: "07a197e07617649edf06a309a9a27ff1"
server_key_version: "encryptionkey@0"
I20260808 04:49:38.637246 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "560e5076c7d3441aa835c4f6b38cb01b"
format_stamp: "Formatted at 2026-08-08 04:49:38 on dist-test-slave-04bb"
server_key: "3b3806cb91fd74cdbb3c213ab5733baf"
server_key_iv: "07a197e07617649edf06a309a9a27ff1"
server_key_version: "encryptionkey@0"
I20260808 04:49:38.641465 20274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 04:49:38.644004 20296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:38.645406 20274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:49:38.645563 20274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "560e5076c7d3441aa835c4f6b38cb01b"
format_stamp: "Formatted at 2026-08-08 04:49:38 on dist-test-slave-04bb"
server_key: "3b3806cb91fd74cdbb3c213ab5733baf"
server_key_iv: "07a197e07617649edf06a309a9a27ff1"
server_key_version: "encryptionkey@0"
I20260808 04:49:38.645676 20274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:38.671510 20274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:38.674942 20274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:38.675156 20274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:38.682680 20274 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33215
I20260808 04:49:38.682701 20388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33215 every 8 connection(s)
I20260808 04:49:38.683619 20274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:38.686156 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 20274
I20260808 04:49:38.686272 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:38.686514 20390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:38.686583 22538 external_mini_cluster.cc:1428] Setting key 11122ce1bbd75ee791160b109f591185
I20260808 04:49:38.693859 20390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b: Bootstrap starting.
I20260808 04:49:38.696511 20390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:38.696945 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:38.688075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32989 (local address 127.0.0.1:33215)
0808 04:49:38.688509 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:38.688550 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:38.688596 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:49:38.689371 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:38.689397 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:38.689835 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:38.690213 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:38.690643 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:38.692095 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:38.693075 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:38.693747 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:38.693914 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:38.694881 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:38.694889 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:38.694909 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:38.695402 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:38.695465 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:38.695494 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:38.695627 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:38.695851 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 04:49:38.695977 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":128,"threads_started":1}
I20260808 04:49:38.697438 20390 log.cc:824] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:38.699623 20390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b: No bootstrap required, opened a new log
I20260808 04:49:38.700083 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:38.703747 20390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560e5076c7d3441aa835c4f6b38cb01b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33215 } }
I20260808 04:49:38.703989 20390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:38.704051 20390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560e5076c7d3441aa835c4f6b38cb01b, State: Initialized, Role: FOLLOWER
I20260808 04:49:38.704553 20390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560e5076c7d3441aa835c4f6b38cb01b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33215 } }
I20260808 04:49:38.704695 20390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:38.704778 20390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:38.704860 20390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:38.705732 20390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560e5076c7d3441aa835c4f6b38cb01b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33215 } }
I20260808 04:49:38.706069 20390 leader_election.cc:302] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 560e5076c7d3441aa835c4f6b38cb01b; no voters:
I20260808 04:49:38.706437 20390 leader_election.cc:288] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:38.706558 20404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:38.706763 20404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [term 1 LEADER]: Becoming Leader. State: Replica: 560e5076c7d3441aa835c4f6b38cb01b, State: Running, Role: LEADER
I20260808 04:49:38.706882 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:38.702150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52901 (local address 127.0.0.1:33215)
0808 04:49:38.702279 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:38.702285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:38.702513 (+ 228us) server_negotiation.cc:407] Connection header received
0808 04:49:38.702591 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:38.702594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:38.702638 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:38.702731 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:38.702948 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:38.703682 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:38.704782 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:38.705029 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:38.705164 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:38.706125 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:38.706129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:38.706132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:38.706147 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:38.706182 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:38.706217 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:38.706319 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:38.706457 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 04:49:38.706492 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:49:38.707074 20404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560e5076c7d3441aa835c4f6b38cb01b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33215 } }
I20260808 04:49:38.707705 20390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:38.708365 20407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 560e5076c7d3441aa835c4f6b38cb01b. Latest consensus state: current_term: 1 leader_uuid: "560e5076c7d3441aa835c4f6b38cb01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560e5076c7d3441aa835c4f6b38cb01b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33215 } } }
I20260808 04:49:38.708541 20407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:38.708978 20415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:38.708936 20406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "560e5076c7d3441aa835c4f6b38cb01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560e5076c7d3441aa835c4f6b38cb01b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33215 } } }
I20260808 04:49:38.710048 20406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:38.711962 20415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:38.717387 20415 catalog_manager.cc:1347] Generated new cluster ID: 84aeb73506c3469295179c49348178d4
I20260808 04:49:38.717450 20415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:38.726287 20415 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:38.727448 20415 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:38.735224 20415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 560e5076c7d3441aa835c4f6b38cb01b: Generated new TSK 0
I20260808 04:49:38.735980 20415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:38.743784 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 20274
W20260808 04:49:38.762691 20256 connection.cc:535] client connection to 127.0.0.1:33215 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33215): Connection reset by peer (error 104)
2026-08-08T04:49:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:38 dist-test-slave-04bb krb5kdc[20432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:38 dist-test-slave-04bb krb5kdc[20432](info): set up 2 sockets
Aug 08 04:49:38 dist-test-slave-04bb krb5kdc[20432](info): commencing operation
krb5kdc: starting...
W20260808 04:49:39.973204 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:40 dist-test-slave-04bb krb5kdc[20432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:40.107781 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37893
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:40.222743 20447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:40.223007 20447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:40.223050 20447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:40.227355 20447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:40.227432 20447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:40.227453 20447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:40.227471 20447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:40.227487 20447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:40.232985 20447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:40.234077 20447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:40.235133 20447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:40.244592 20462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:40.244652 20464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:40.244619 20460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:40.244903 20447 server_base.cc:1034] running on GCE node
I20260808 04:49:40.245461 20447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:40.246640 20447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:40.247843 20447 hybrid_clock.cc:648] HybridClock initialized: now 1786164580247809 us; error 44 us; skew 500 ppm
Aug 08 04:49:40 dist-test-slave-04bb krb5kdc[20432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:40.254654 20447 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:40.255620 20447 webserver.cc:460] Webserver started at http://127.0.0.1:44201/ using document root <none> and password file <none>
I20260808 04:49:40.256201 20447 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:40.256294 20447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:40.256603 20447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:40.258657 20447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "6a58f63a922c4006a4e6ed97d7199f51"
format_stamp: "Formatted at 2026-08-08 04:49:40 on dist-test-slave-04bb"
server_key: "3d5b529a83c8c985609088b8d10e785f"
server_key_iv: "96288667f47c9e44ae2033089d9f6567"
server_key_version: "encryptionkey@0"
I20260808 04:49:40.259236 20447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "6a58f63a922c4006a4e6ed97d7199f51"
format_stamp: "Formatted at 2026-08-08 04:49:40 on dist-test-slave-04bb"
server_key: "3d5b529a83c8c985609088b8d10e785f"
server_key_iv: "96288667f47c9e44ae2033089d9f6567"
server_key_version: "encryptionkey@0"
I20260808 04:49:40.263147 20447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 04:49:40.265651 20475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:40.266885 20447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:40.267030 20447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "6a58f63a922c4006a4e6ed97d7199f51"
format_stamp: "Formatted at 2026-08-08 04:49:40 on dist-test-slave-04bb"
server_key: "3d5b529a83c8c985609088b8d10e785f"
server_key_iv: "96288667f47c9e44ae2033089d9f6567"
server_key_version: "encryptionkey@0"
I20260808 04:49:40.267140 20447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:40.271725 20447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:40.275049 20447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:40.275238 20447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:40.282752 20447 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37893
I20260808 04:49:40.282768 20551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37893 every 8 connection(s)
I20260808 04:49:40.283668 20447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:40.285043 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 20447
I20260808 04:49:40.285193 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:40.285493 22538 external_mini_cluster.cc:1428] Setting key 177178b0a9e2e3af4abaa292fb245275
I20260808 04:49:40.286942 20554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:40.291268 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:40.286957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34359 (local address 127.0.0.1:37893)
0808 04:49:40.287419 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:40.287442 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:40.287486 (+ 44us) server_negotiation.cc:407] Connection header received
0808 04:49:40.288321 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:40.288344 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:40.288732 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:40.289103 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:40.289807 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:40.289820 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:40.289835 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:40.289847 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:40.289910 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:40.289938 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:40.290079 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:40.290163 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 04:49:40.290319 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":115,"threads_started":1}
I20260808 04:49:40.294426 20554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51: Bootstrap starting.
I20260808 04:49:40.294443 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:40.296846 20554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:40.297847 20554 log.cc:824] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:40.298403 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:40.296593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44419 (local address 127.0.0.1:37893)
0808 04:49:40.296733 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:40.296738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:40.297008 (+ 270us) server_negotiation.cc:407] Connection header received
0808 04:49:40.297104 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:40.297106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:40.297138 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:40.297203 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:40.297927 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:40.297930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:40.297932 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:40.297978 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:40.298010 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:40.298023 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:40.298103 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:40.298228 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 04:49:40.298281 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 04:49:40.299974 20554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51: No bootstrap required, opened a new log
I20260808 04:49:40.303871 20554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a58f63a922c4006a4e6ed97d7199f51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260808 04:49:40.304124 20554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:40.304209 20554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a58f63a922c4006a4e6ed97d7199f51, State: Initialized, Role: FOLLOWER
I20260808 04:49:40.304675 20554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a58f63a922c4006a4e6ed97d7199f51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260808 04:49:40.304831 20554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:40.304921 20554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:40.305040 20554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:40.305940 20554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a58f63a922c4006a4e6ed97d7199f51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260808 04:49:40.306308 20554 leader_election.cc:302] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a58f63a922c4006a4e6ed97d7199f51; no voters:
I20260808 04:49:40.306610 20554 leader_election.cc:288] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:40.306704 20570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:40.307005 20570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [term 1 LEADER]: Becoming Leader. State: Replica: 6a58f63a922c4006a4e6ed97d7199f51, State: Running, Role: LEADER
I20260808 04:49:40.307412 20570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a58f63a922c4006a4e6ed97d7199f51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260808 04:49:40.307895 20554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:40.309120 20571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a58f63a922c4006a4e6ed97d7199f51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a58f63a922c4006a4e6ed97d7199f51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } } }
I20260808 04:49:40.309288 20571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:40.309690 20572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a58f63a922c4006a4e6ed97d7199f51. Latest consensus state: current_term: 1 leader_uuid: "6a58f63a922c4006a4e6ed97d7199f51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a58f63a922c4006a4e6ed97d7199f51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } } }
I20260808 04:49:40.309863 20572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:40.310503 20581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:40.313344 20581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:40.319131 20581 catalog_manager.cc:1347] Generated new cluster ID: 28d61a1441a54a4fba1519dd8a2e9cff
I20260808 04:49:40.319188 20581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:40.331018 20581 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:40.331329 20581 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:40.341310 20581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a58f63a922c4006a4e6ed97d7199f51: Generated new TSK 0
I20260808 04:49:40.342063 20581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:40.349459 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 20447
2026-08-08T04:49:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1604 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:40 dist-test-slave-04bb krb5kdc[20596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:40 dist-test-slave-04bb krb5kdc[20596](info): set up 2 sockets
Aug 08 04:49:40 dist-test-slave-04bb krb5kdc[20596](info): commencing operation
krb5kdc: starting...
W20260808 04:49:41.598134 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:41 dist-test-slave-04bb krb5kdc[20596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:41.723610 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:41.841053 20614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:41.841315 20614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:41.841355 20614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:41.845247 20614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:41.845306 20614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:41.845326 20614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:41.845342 20614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:41.845357 20614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:41.850507 20614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:41.851516 20614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:41.852504 20614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:41.861268 20622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:41.861289 20623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:41.861289 20625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:41.861621 20614 server_base.cc:1034] running on GCE node
I20260808 04:49:41.862205 20614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:41.863360 20614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:41.864589 20614 hybrid_clock.cc:648] HybridClock initialized: now 1786164581864569 us; error 30 us; skew 500 ppm
Aug 08 04:49:41 dist-test-slave-04bb krb5kdc[20596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164581, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:41.867301 20614 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:41.868211 20614 webserver.cc:460] Webserver started at http://127.0.0.1:38415/ using document root <none> and password file <none>
I20260808 04:49:41.868861 20614 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:41.868956 20614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:41.869203 20614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:41.871292 20614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "54c6faf2d90c44c694654ca64cd3eab5"
format_stamp: "Formatted at 2026-08-08 04:49:41 on dist-test-slave-04bb"
server_key: "b87627c46dba898bfbde014cf95f609f"
server_key_iv: "4ed5cc4d9d5296e4cbd0f16f810731be"
server_key_version: "encryptionkey@0"
I20260808 04:49:41.871878 20614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "54c6faf2d90c44c694654ca64cd3eab5"
format_stamp: "Formatted at 2026-08-08 04:49:41 on dist-test-slave-04bb"
server_key: "b87627c46dba898bfbde014cf95f609f"
server_key_iv: "4ed5cc4d9d5296e4cbd0f16f810731be"
server_key_version: "encryptionkey@0"
I20260808 04:49:41.875698 20614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 04:49:41.878216 20639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:41.879483 20614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:41.879632 20614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "54c6faf2d90c44c694654ca64cd3eab5"
format_stamp: "Formatted at 2026-08-08 04:49:41 on dist-test-slave-04bb"
server_key: "b87627c46dba898bfbde014cf95f609f"
server_key_iv: "4ed5cc4d9d5296e4cbd0f16f810731be"
server_key_version: "encryptionkey@0"
I20260808 04:49:41.879750 20614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:41.894667 20614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:41.897820 20614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:41.898039 20614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:41.905474 20614 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38747
I20260808 04:49:41.905486 20707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38747 every 8 connection(s)
I20260808 04:49:41.906566 20614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:41.909514 20708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:41.910250 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 20614
I20260808 04:49:41.910400 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:41.910748 22538 external_mini_cluster.cc:1428] Setting key 925c0dee4790a3a1d1f42b66d3754ab5
I20260808 04:49:41.916137 20708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5: Bootstrap starting.
I20260808 04:49:41.918654 20708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:41.919420 20708 log.cc:824] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:49:41 dist-test-slave-04bb krb5kdc[20596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 04:49:41.922078 20708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5: No bootstrap required, opened a new log
I20260808 04:49:41.926367 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:41.912125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37743 (local address 127.0.0.1:38747)
0808 04:49:41.912610 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:41.912625 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:41.912914 (+ 289us) server_negotiation.cc:407] Connection header received
0808 04:49:41.913665 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:41.913682 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:41.914032 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:41.914316 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:41.915237 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:41.916186 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:41.917158 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:41.917630 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:41.917824 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:41.920564 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:41.920571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:41.920588 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:41.920617 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:41.922926 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:41.923431 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:41.923438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:41.923444 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:41.923599 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:41.923898 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:41.923902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:41.923904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:41.924491 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:41.924788 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:41.925014 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 04:49:41.925347 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":162,"threads_started":1}
I20260808 04:49:41.927191 20708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c6faf2d90c44c694654ca64cd3eab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38747 } }
I20260808 04:49:41.927482 20708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:41.927549 20708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c6faf2d90c44c694654ca64cd3eab5, State: Initialized, Role: FOLLOWER
I20260808 04:49:41.928061 20708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c6faf2d90c44c694654ca64cd3eab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38747 } }
I20260808 04:49:41.928196 20708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:41.928295 20708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:41.928422 20708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:41.929580 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:41.929600 20708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c6faf2d90c44c694654ca64cd3eab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38747 } }
I20260808 04:49:41.929960 20708 leader_election.cc:302] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54c6faf2d90c44c694654ca64cd3eab5; no voters:
I20260808 04:49:41.930302 20708 leader_election.cc:288] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:41.930418 20713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:41.930665 20713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [term 1 LEADER]: Becoming Leader. State: Replica: 54c6faf2d90c44c694654ca64cd3eab5, State: Running, Role: LEADER
I20260808 04:49:41.930995 20713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c6faf2d90c44c694654ca64cd3eab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38747 } }
I20260808 04:49:41.931686 20708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:41.932297 20715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54c6faf2d90c44c694654ca64cd3eab5. Latest consensus state: current_term: 1 leader_uuid: "54c6faf2d90c44c694654ca64cd3eab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c6faf2d90c44c694654ca64cd3eab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38747 } } }
I20260808 04:49:41.932432 20715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:41.932816 20714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54c6faf2d90c44c694654ca64cd3eab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c6faf2d90c44c694654ca64cd3eab5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38747 } } }
I20260808 04:49:41.932945 20714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5 [sys.catalog]: This master's current role is: LEADER
W20260808 04:49:41.937625 20744 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:49:41.937740 20744 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:49:41.937806 20725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:41.938798 20725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:41.942895 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:41.931991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49371 (local address 127.0.0.1:38747)
0808 04:49:41.932133 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:41.932138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:41.933579 (+ 1441us) server_negotiation.cc:407] Connection header received
0808 04:49:41.933778 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:41.933785 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:41.933850 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:41.933945 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:41.934763 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:41.935384 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:41.936483 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:41.936883 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:41.937001 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:41.938715 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:41.938720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:41.938723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:41.938751 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:41.940655 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:41.941227 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:41.941234 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:41.941237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:41.941335 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:41.941764 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:41.941770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:41.941775 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:41.942053 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:41.942195 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:41.942567 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 04:49:41.942713 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 04:49:41.944603 20725 catalog_manager.cc:1347] Generated new cluster ID: 7acae56add8548829536eb66c0912c16
I20260808 04:49:41.944674 20725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:41.958935 20725 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:41.960075 20725 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:41.966403 20725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54c6faf2d90c44c694654ca64cd3eab5: Generated new TSK 0
I20260808 04:49:41.967130 20725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:41.971148 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 20614
2026-08-08T04:49:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:42 dist-test-slave-04bb krb5kdc[20753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:42 dist-test-slave-04bb krb5kdc[20753](info): set up 2 sockets
Aug 08 04:49:42 dist-test-slave-04bb krb5kdc[20753](info): commencing operation
krb5kdc: starting...
W20260808 04:49:43.243454 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:43 dist-test-slave-04bb krb5kdc[20753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:43.368397 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:43.481572 20776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:43.481844 20776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:43.481886 20776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:43.485853 20776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:43.485906 20776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:43.485925 20776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:43.485940 20776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:43.485955 20776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:43.491068 20776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:43.492070 20776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:43.493132 20776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:43.501726 20789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:43.501700 20793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:43.501741 20791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:43.502053 20776 server_base.cc:1034] running on GCE node
I20260808 04:49:43.502677 20776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:43.503701 20776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:43.504861 20776 hybrid_clock.cc:648] HybridClock initialized: now 1786164583504842 us; error 30 us; skew 500 ppm
Aug 08 04:49:43 dist-test-slave-04bb krb5kdc[20753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164583, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:43.507562 20776 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:43.508466 20776 webserver.cc:460] Webserver started at http://127.0.0.1:42759/ using document root <none> and password file <none>
I20260808 04:49:43.509048 20776 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:43.509143 20776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:43.509433 20776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:43.511449 20776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "41c67cfd1b7c4b2fa788a3219e514485"
format_stamp: "Formatted at 2026-08-08 04:49:43 on dist-test-slave-04bb"
server_key: "397c9d3c4de8ff4832552d6d041f6875"
server_key_iv: "cd68c8527d6c1c4fd5eed623bc821071"
server_key_version: "encryptionkey@0"
I20260808 04:49:43.512064 20776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "41c67cfd1b7c4b2fa788a3219e514485"
format_stamp: "Formatted at 2026-08-08 04:49:43 on dist-test-slave-04bb"
server_key: "397c9d3c4de8ff4832552d6d041f6875"
server_key_iv: "cd68c8527d6c1c4fd5eed623bc821071"
server_key_version: "encryptionkey@0"
I20260808 04:49:43.515936 20776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 04:49:43.518390 20804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:43.519618 20776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:43.519764 20776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "41c67cfd1b7c4b2fa788a3219e514485"
format_stamp: "Formatted at 2026-08-08 04:49:43 on dist-test-slave-04bb"
server_key: "397c9d3c4de8ff4832552d6d041f6875"
server_key_iv: "cd68c8527d6c1c4fd5eed623bc821071"
server_key_version: "encryptionkey@0"
I20260808 04:49:43.519898 20776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:43.547806 20776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:43.550818 20776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:43.550982 20776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:43.558282 20776 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35499
I20260808 04:49:43.558300 20877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35499 every 8 connection(s)
I20260808 04:49:43.559281 20776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:43.562245 20878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:43.564910 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 20776
I20260808 04:49:43.565081 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:43.565367 22538 external_mini_cluster.cc:1428] Setting key 1356b71667c2d562187f07472e35425f
I20260808 04:49:43.568601 20878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485: Bootstrap starting.
I20260808 04:49:43.571215 20878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:43.572126 20878 log.cc:824] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:43.574179 20878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485: No bootstrap required, opened a new log
I20260808 04:49:43.574389 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:43.566713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57737 (local address 127.0.0.1:35499)
0808 04:49:43.567136 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:43.567152 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:43.567189 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:49:43.567805 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:43.567822 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:43.568181 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:43.568467 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:43.568912 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:43.569862 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:43.571061 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:43.571578 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:43.571696 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:43.572725 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:43.572730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:43.572744 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:43.573120 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:43.573173 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:43.573197 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:43.573320 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:43.573434 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 04:49:43.573640 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":132,"threads_started":1}
I20260808 04:49:43.577266 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:43.577961 20878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c67cfd1b7c4b2fa788a3219e514485" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35499 } }
I20260808 04:49:43.578189 20878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:43.578251 20878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c67cfd1b7c4b2fa788a3219e514485, State: Initialized, Role: FOLLOWER
I20260808 04:49:43.578759 20878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c67cfd1b7c4b2fa788a3219e514485" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35499 } }
I20260808 04:49:43.578891 20878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:43.578987 20878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:43.579141 20878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:43.580392 20878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c67cfd1b7c4b2fa788a3219e514485" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35499 } }
I20260808 04:49:43.580771 20878 leader_election.cc:302] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41c67cfd1b7c4b2fa788a3219e514485; no voters:
I20260808 04:49:43.581158 20878 leader_election.cc:288] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:43.581248 20892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:43.581490 20892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [term 1 LEADER]: Becoming Leader. State: Replica: 41c67cfd1b7c4b2fa788a3219e514485, State: Running, Role: LEADER
I20260808 04:49:43.581846 20892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c67cfd1b7c4b2fa788a3219e514485" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35499 } }
I20260808 04:49:43.582481 20878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:43.583789 20894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41c67cfd1b7c4b2fa788a3219e514485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c67cfd1b7c4b2fa788a3219e514485" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35499 } } }
I20260808 04:49:43.583925 20894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:43.584300 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:43.579388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54197 (local address 127.0.0.1:35499)
0808 04:49:43.579527 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:43.579531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:43.579773 (+ 242us) server_negotiation.cc:407] Connection header received
0808 04:49:43.579886 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:43.579888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:43.579925 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:43.579988 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:43.580318 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:43.580908 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:43.582067 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:43.582383 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:43.582494 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:43.583443 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:43.583448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:43.583451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:43.583467 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:43.583517 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:43.583535 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:43.583612 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:43.584078 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 04:49:43.584136 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:49:43.584327 20895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41c67cfd1b7c4b2fa788a3219e514485. Latest consensus state: current_term: 1 leader_uuid: "41c67cfd1b7c4b2fa788a3219e514485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c67cfd1b7c4b2fa788a3219e514485" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35499 } } }
I20260808 04:49:43.584407 20895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485 [sys.catalog]: This master's current role is: LEADER
W20260808 04:49:43.587473 20915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:49:43.587565 20915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:49:43.587653 20903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:43.588491 20903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:43.593585 20903 catalog_manager.cc:1347] Generated new cluster ID: 740baf0bb0cf4ef095ead0acbaedea57
I20260808 04:49:43.593647 20903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:43.605593 20903 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:43.606487 20903 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:43.616125 20903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41c67cfd1b7c4b2fa788a3219e514485: Generated new TSK 0
I20260808 04:49:43.616878 20903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:43.622306 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 20776
2026-08-08T04:49:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1663 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:43 dist-test-slave-04bb krb5kdc[20922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:43 dist-test-slave-04bb krb5kdc[20922](info): set up 2 sockets
Aug 08 04:49:43 dist-test-slave-04bb krb5kdc[20922](info): commencing operation
krb5kdc: starting...
W20260808 04:49:44.877254 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:44 dist-test-slave-04bb krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:45.004227 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38301
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:45.117270 20939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:45.117509 20939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:45.117549 20939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:45.121512 20939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:45.121563 20939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:45.121582 20939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:45.121599 20939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:45.121614 20939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:45.126875 20939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:45.127797 20939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:45.128816 20939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:49:45.137244 20939 server_base.cc:1034] running on GCE node
W20260808 04:49:45.137094 20947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:45.137094 20951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:45.137100 20948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:45.138008 20939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:45.139120 20939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:45.140364 20939 hybrid_clock.cc:648] HybridClock initialized: now 1786164585140345 us; error 32 us; skew 500 ppm
Aug 08 04:49:45 dist-test-slave-04bb krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164585, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:45.143258 20939 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:45.144147 20939 webserver.cc:460] Webserver started at http://127.0.0.1:41979/ using document root <none> and password file <none>
I20260808 04:49:45.144806 20939 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:45.144901 20939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:45.145139 20939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:45.147195 20939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "14806ff663584e3bb68298e0da440be8"
format_stamp: "Formatted at 2026-08-08 04:49:45 on dist-test-slave-04bb"
server_key: "1f2acfc4298605eb51453ccbfcfa9834"
server_key_iv: "ca7cd942a8961f37d72b859c723ad641"
server_key_version: "encryptionkey@0"
I20260808 04:49:45.147783 20939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "14806ff663584e3bb68298e0da440be8"
format_stamp: "Formatted at 2026-08-08 04:49:45 on dist-test-slave-04bb"
server_key: "1f2acfc4298605eb51453ccbfcfa9834"
server_key_iv: "ca7cd942a8961f37d72b859c723ad641"
server_key_version: "encryptionkey@0"
I20260808 04:49:45.151559 20939 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 04:49:45.154059 20968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:45.155287 20939 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:45.155450 20939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "14806ff663584e3bb68298e0da440be8"
format_stamp: "Formatted at 2026-08-08 04:49:45 on dist-test-slave-04bb"
server_key: "1f2acfc4298605eb51453ccbfcfa9834"
server_key_iv: "ca7cd942a8961f37d72b859c723ad641"
server_key_version: "encryptionkey@0"
I20260808 04:49:45.155566 20939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:45.160178 20939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:45.163460 20939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:45.163655 20939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:45.170763 20939 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38301
I20260808 04:49:45.170792 21041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38301 every 8 connection(s)
I20260808 04:49:45.171685 20939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:45.174494 21042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:45.180025 21042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8: Bootstrap starting.
I20260808 04:49:45.180424 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 20939
I20260808 04:49:45.180612 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:45.180895 22538 external_mini_cluster.cc:1428] Setting key 3500e5ee03ac2fc17b6f16e1d6d0b21e
I20260808 04:49:45.182823 21042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:45.183710 21042 log.cc:824] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:45.185803 21042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8: No bootstrap required, opened a new log
I20260808 04:49:45.186512 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:45.182256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44177 (local address 127.0.0.1:38301)
0808 04:49:45.182748 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:45.182772 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:45.182820 (+ 48us) server_negotiation.cc:407] Connection header received
0808 04:49:45.183683 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:45.183706 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:45.184073 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:45.184444 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:45.185263 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:45.185269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:45.185291 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:45.185302 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:45.185362 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:45.185382 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:45.185504 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:45.185621 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 04:49:45.185783 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":118,"threads_started":1}
I20260808 04:49:45.189294 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:45.189569 21042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14806ff663584e3bb68298e0da440be8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } }
I20260808 04:49:45.189770 21042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:45.189846 21042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14806ff663584e3bb68298e0da440be8, State: Initialized, Role: FOLLOWER
I20260808 04:49:45.190315 21042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14806ff663584e3bb68298e0da440be8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } }
I20260808 04:49:45.190443 21042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:45.190528 21042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:45.190640 21042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:45.191644 21042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14806ff663584e3bb68298e0da440be8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } }
I20260808 04:49:45.192047 21042 leader_election.cc:302] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14806ff663584e3bb68298e0da440be8; no voters:
I20260808 04:49:45.192468 21042 leader_election.cc:288] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:45.192595 21059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:45.192835 21059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [term 1 LEADER]: Becoming Leader. State: Replica: 14806ff663584e3bb68298e0da440be8, State: Running, Role: LEADER
I20260808 04:49:45.193210 21059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14806ff663584e3bb68298e0da440be8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } }
I20260808 04:49:45.193339 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:45.191427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34145 (local address 127.0.0.1:38301)
0808 04:49:45.191526 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:45.191530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:45.191716 (+ 186us) server_negotiation.cc:407] Connection header received
0808 04:49:45.191885 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:45.191888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:45.191917 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:45.191983 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:45.192909 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:45.192913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:45.192915 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:45.192927 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:45.192952 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:45.192965 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:45.193020 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:45.193191 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 04:49:45.193229 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 04:49:45.193859 21042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:45.194908 21059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14806ff663584e3bb68298e0da440be8. Latest consensus state: current_term: 1 leader_uuid: "14806ff663584e3bb68298e0da440be8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14806ff663584e3bb68298e0da440be8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } } }
I20260808 04:49:45.195091 21059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:45.195334 21060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14806ff663584e3bb68298e0da440be8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14806ff663584e3bb68298e0da440be8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } } }
I20260808 04:49:45.195441 21060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:45.195550 21071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:45.199210 21071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:45.205106 21071 catalog_manager.cc:1347] Generated new cluster ID: fa4c57dde2dd4de4b4d488ab99529c83
I20260808 04:49:45.205170 21071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:45.214887 21071 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:45.215185 21071 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:45.219761 21071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14806ff663584e3bb68298e0da440be8: Generated new TSK 0
I20260808 04:49:45.220508 21071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:45.227005 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 20939
2026-08-08T04:49:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:45 dist-test-slave-04bb krb5kdc[21089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:45 dist-test-slave-04bb krb5kdc[21089](info): set up 2 sockets
Aug 08 04:49:45 dist-test-slave-04bb krb5kdc[21089](info): commencing operation
krb5kdc: starting...
W20260808 04:49:46.468816 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:46 dist-test-slave-04bb krb5kdc[21089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:46.593485 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:46.705399 21106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:46.705644 21106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:46.705683 21106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:46.709604 21106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:46.709656 21106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:46.709674 21106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:46.709692 21106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:46.709707 21106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:46.714749 21106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:46.715687 21106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:46.716792 21106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:46.725327 21117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:46.725468 21106 server_base.cc:1034] running on GCE node
W20260808 04:49:46.725394 21115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:46.725394 21114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:46.726228 21106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:46.727311 21106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:46.728489 21106 hybrid_clock.cc:648] HybridClock initialized: now 1786164586728470 us; error 31 us; skew 500 ppm
Aug 08 04:49:46 dist-test-slave-04bb krb5kdc[21089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:46.731235 21106 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:46.732115 21106 webserver.cc:460] Webserver started at http://127.0.0.1:37557/ using document root <none> and password file <none>
I20260808 04:49:46.732751 21106 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:46.732869 21106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:46.733101 21106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:46.735124 21106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "d764d04cdafb4c5db50e4d1d11a1748a"
format_stamp: "Formatted at 2026-08-08 04:49:46 on dist-test-slave-04bb"
server_key: "1397448f40120aeb176d800deaaa47ae"
server_key_iv: "c7443a49a9df4dac0de4ba04c4889407"
server_key_version: "encryptionkey@0"
I20260808 04:49:46.735710 21106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "d764d04cdafb4c5db50e4d1d11a1748a"
format_stamp: "Formatted at 2026-08-08 04:49:46 on dist-test-slave-04bb"
server_key: "1397448f40120aeb176d800deaaa47ae"
server_key_iv: "c7443a49a9df4dac0de4ba04c4889407"
server_key_version: "encryptionkey@0"
I20260808 04:49:46.739470 21106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 04:49:46.741935 21125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:46.743134 21106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:46.743279 21106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "d764d04cdafb4c5db50e4d1d11a1748a"
format_stamp: "Formatted at 2026-08-08 04:49:46 on dist-test-slave-04bb"
server_key: "1397448f40120aeb176d800deaaa47ae"
server_key_iv: "c7443a49a9df4dac0de4ba04c4889407"
server_key_version: "encryptionkey@0"
I20260808 04:49:46.743397 21106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:46.759582 21106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:46.762338 21106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:46.762545 21106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:46.769618 21106 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38305
I20260808 04:49:46.769629 21195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38305 every 8 connection(s)
I20260808 04:49:46.770577 21106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:46.773542 21196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:46.778964 21196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a: Bootstrap starting.
I20260808 04:49:46.779378 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 21106
I20260808 04:49:46.779538 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:46.779835 22538 external_mini_cluster.cc:1428] Setting key 39bd6ea56a3820c13d47aa27c0806d84
I20260808 04:49:46.781808 21196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:46.782554 21196 log.cc:824] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:46.784866 21196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a: No bootstrap required, opened a new log
Aug 08 04:49:46 dist-test-slave-04bb krb5kdc[21089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 04:49:46.789470 21196 raft_consensus.cc:348] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d764d04cdafb4c5db50e4d1d11a1748a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260808 04:49:46.789719 21196 raft_consensus.cc:374] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:46.789796 21196 raft_consensus.cc:729] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d764d04cdafb4c5db50e4d1d11a1748a, State: Initialized, Role: FOLLOWER
I20260808 04:49:46.790319 21196 consensus_queue.cc:260] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d764d04cdafb4c5db50e4d1d11a1748a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260808 04:49:46.790511 21196 raft_consensus.cc:388] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:46.790588 21196 raft_consensus.cc:482] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:46.790699 21196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:46.791905 21196 raft_consensus.cc:504] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d764d04cdafb4c5db50e4d1d11a1748a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260808 04:49:46.792285 21196 leader_election.cc:302] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d764d04cdafb4c5db50e4d1d11a1748a; no voters:
I20260808 04:49:46.792743 21196 leader_election.cc:288] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:46.792814 21201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:46.793061 21201 raft_consensus.cc:686] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [term 1 LEADER]: Becoming Leader. State: Replica: d764d04cdafb4c5db50e4d1d11a1748a, State: Running, Role: LEADER
I20260808 04:49:46.793401 21201 consensus_queue.cc:237] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d764d04cdafb4c5db50e4d1d11a1748a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } }
I20260808 04:49:46.794111 21196 sys_catalog.cc:564] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:46.794651 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d764d04cdafb4c5db50e4d1d11a1748a. Latest consensus state: current_term: 1 leader_uuid: "d764d04cdafb4c5db50e4d1d11a1748a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d764d04cdafb4c5db50e4d1d11a1748a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } } }
I20260808 04:49:46.794767 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:46.794987 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d764d04cdafb4c5db50e4d1d11a1748a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d764d04cdafb4c5db50e4d1d11a1748a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38305 } } }
I20260808 04:49:46.795092 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:46.795255 21209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:46.795981 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:46.781295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:33043 (local address 127.0.0.1:38305)
0808 04:49:46.781725 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:46.781748 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:46.781784 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:49:46.782468 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:46.782485 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:46.782915 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:46.783306 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:46.784187 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:46.785271 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:46.786458 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:46.786933 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:46.787051 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:46.790041 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:46.790047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:46.790064 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:46.790096 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:46.792211 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:46.792821 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:46.792845 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:46.792852 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:46.792992 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:46.793378 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:46.793382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:46.793384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:46.793864 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:46.794103 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:46.794717 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 04:49:46.794991 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":134,"threads_started":1}
I20260808 04:49:46.799104 21209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:46.799361 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:46.806260 21209 catalog_manager.cc:1347] Generated new cluster ID: b825681b93814775b55b9065a069ea44
I20260808 04:49:46.806334 21209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:46.810425 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:46.801482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:32807 (local address 127.0.0.1:38305)
0808 04:49:46.801595 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:46.801602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:46.801913 (+ 311us) server_negotiation.cc:407] Connection header received
0808 04:49:46.802044 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:46.802047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:46.802099 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:46.802184 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:46.802830 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:46.803434 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:46.804710 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:46.804962 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:46.805079 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:46.806617 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:46.806623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:46.806627 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:46.806656 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:46.808265 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:46.808860 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:46.808864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:46.808866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:46.808950 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:46.809322 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:46.809326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:46.809327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:46.809623 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:46.809791 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:46.810149 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 04:49:46.810242 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 04:49:46.817828 21209 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:46.818709 21209 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:46.827451 21209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d764d04cdafb4c5db50e4d1d11a1748a: Generated new TSK 0
I20260808 04:49:46.828255 21209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:46.840715 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 21106
W20260808 04:49:46.853479 21087 connection.cc:535] client connection to 127.0.0.1:38305 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38305): Connection reset by peer (error 104)
2026-08-08T04:49:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:46 dist-test-slave-04bb krb5kdc[21244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:46 dist-test-slave-04bb krb5kdc[21244](info): set up 2 sockets
Aug 08 04:49:46 dist-test-slave-04bb krb5kdc[21244](info): commencing operation
krb5kdc: starting...
W20260808 04:49:48.126513 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:48 dist-test-slave-04bb krb5kdc[21244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:48.253685 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:48.367233 21265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:48.367478 21265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:48.367518 21265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:48.371572 21265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:48.371625 21265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:48.371644 21265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:48.371659 21265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:48.371673 21265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:48.376688 21265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:48.377626 21265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:48.378607 21265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:48.387158 21277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:48.387181 21275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:48.387184 21272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:48.387435 21265 server_base.cc:1034] running on GCE node
I20260808 04:49:48.388108 21265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:48.389252 21265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:48.390422 21265 hybrid_clock.cc:648] HybridClock initialized: now 1786164588390403 us; error 32 us; skew 500 ppm
Aug 08 04:49:48 dist-test-slave-04bb krb5kdc[21244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:48.393208 21265 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:48.394040 21265 webserver.cc:460] Webserver started at http://127.0.0.1:39663/ using document root <none> and password file <none>
I20260808 04:49:48.394583 21265 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:48.394641 21265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:48.394814 21265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:48.396858 21265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "85481ab8b56541c2a4337184c9275d9d"
format_stamp: "Formatted at 2026-08-08 04:49:48 on dist-test-slave-04bb"
server_key: "5ccd44ce4095e2dbbad1f48750083595"
server_key_iv: "f8fa555742098d4d5c18902f4b05ffe9"
server_key_version: "encryptionkey@0"
I20260808 04:49:48.397347 21265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "85481ab8b56541c2a4337184c9275d9d"
format_stamp: "Formatted at 2026-08-08 04:49:48 on dist-test-slave-04bb"
server_key: "5ccd44ce4095e2dbbad1f48750083595"
server_key_iv: "f8fa555742098d4d5c18902f4b05ffe9"
server_key_version: "encryptionkey@0"
I20260808 04:49:48.400992 21265 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:49:48.403226 21289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:48.404431 21265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:48.404620 21265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "85481ab8b56541c2a4337184c9275d9d"
format_stamp: "Formatted at 2026-08-08 04:49:48 on dist-test-slave-04bb"
server_key: "5ccd44ce4095e2dbbad1f48750083595"
server_key_iv: "f8fa555742098d4d5c18902f4b05ffe9"
server_key_version: "encryptionkey@0"
I20260808 04:49:48.404716 21265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:48.418095 21265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:48.420902 21265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:48.421095 21265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:48.427984 21265 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44859
I20260808 04:49:48.427996 21361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44859 every 8 connection(s)
I20260808 04:49:48.428922 21265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:48.429885 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 21265
I20260808 04:49:48.430037 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:48.430337 22538 external_mini_cluster.cc:1428] Setting key 76e76ee46abfc8f190fbdead7a221fbf
I20260808 04:49:48.432251 21363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:48.439002 21363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d: Bootstrap starting.
I20260808 04:49:48.439594 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:48.431675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:45133 (local address 127.0.0.1:44859)
0808 04:49:48.432092 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:48.432109 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:48.432143 (+ 34us) server_negotiation.cc:407] Connection header received
0808 04:49:48.432861 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:48.432878 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:48.433201 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:48.433469 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:48.433865 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:48.434830 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:48.436070 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:48.436510 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:48.436699 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:48.437817 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:48.437825 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:48.437841 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:48.438236 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:48.438290 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:48.438314 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:48.438416 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:48.438639 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 04:49:48.438818 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":143,"threads_started":1}
I20260808 04:49:48.441489 21363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:48.442350 21363 log.cc:824] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:48.442685 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:48.444650 21363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d: No bootstrap required, opened a new log
I20260808 04:49:48.449095 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:48.444907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:56413 (local address 127.0.0.1:44859)
0808 04:49:48.445041 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:48.445046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:48.445273 (+ 227us) server_negotiation.cc:407] Connection header received
0808 04:49:48.445395 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:48.445397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:48.445433 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:48.445495 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:48.445814 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:48.446277 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:48.447454 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:48.447656 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:48.447724 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:48.448613 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:48.448616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:48.448617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:48.448629 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:48.448650 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:48.448660 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:48.448729 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:48.448911 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 04:49:48.448961 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 04:49:48.449276 21363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85481ab8b56541c2a4337184c9275d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44859 } }
I20260808 04:49:48.449540 21363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:48.449615 21363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85481ab8b56541c2a4337184c9275d9d, State: Initialized, Role: FOLLOWER
I20260808 04:49:48.450073 21363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85481ab8b56541c2a4337184c9275d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44859 } }
I20260808 04:49:48.450235 21363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:48.450304 21363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:48.450402 21363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:48.451246 21363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85481ab8b56541c2a4337184c9275d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44859 } }
I20260808 04:49:48.451592 21363 leader_election.cc:302] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85481ab8b56541c2a4337184c9275d9d; no voters:
I20260808 04:49:48.451912 21363 leader_election.cc:288] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:48.452039 21377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:48.452276 21377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [term 1 LEADER]: Becoming Leader. State: Replica: 85481ab8b56541c2a4337184c9275d9d, State: Running, Role: LEADER
I20260808 04:49:48.452675 21377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85481ab8b56541c2a4337184c9275d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44859 } }
I20260808 04:49:48.453132 21363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:48.454384 21380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85481ab8b56541c2a4337184c9275d9d. Latest consensus state: current_term: 1 leader_uuid: "85481ab8b56541c2a4337184c9275d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85481ab8b56541c2a4337184c9275d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44859 } } }
I20260808 04:49:48.454401 21379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85481ab8b56541c2a4337184c9275d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85481ab8b56541c2a4337184c9275d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44859 } } }
I20260808 04:49:48.454531 21380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:48.454531 21379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:48.454984 21392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:48.458205 21392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:48.463872 21392 catalog_manager.cc:1347] Generated new cluster ID: 116caf412e7c4f4884b5557abfd12f51
I20260808 04:49:48.463946 21392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:48.487185 21392 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:48.488368 21392 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:48.495807 21392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85481ab8b56541c2a4337184c9275d9d: Generated new TSK 0
I20260808 04:49:48.496567 21392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:48.505609 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 21265
W20260808 04:49:48.520910 21242 connection.cc:535] client connection to 127.0.0.1:44859 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44859): Connection reset by peer (error 104)
2026-08-08T04:49:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1667 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:48 dist-test-slave-04bb krb5kdc[21407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:48 dist-test-slave-04bb krb5kdc[21407](info): set up 2 sockets
Aug 08 04:49:48 dist-test-slave-04bb krb5kdc[21407](info): commencing operation
krb5kdc: starting...
W20260808 04:49:49.739236 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:49 dist-test-slave-04bb krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:49.868131 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:49.982920 21421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:49.983196 21421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:49.983237 21421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:49.987339 21421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:49.987398 21421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:49.987418 21421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:49.987434 21421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:49.987449 21421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:49.992779 21421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:49.993892 21421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:49.994978 21421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:49:50.004124 21421 server_base.cc:1034] running on GCE node
W20260808 04:49:50.003983 21435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:50.003983 21433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:50.004006 21431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:50.004891 21421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:50.006065 21421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:50.007251 21421 hybrid_clock.cc:648] HybridClock initialized: now 1786164590007211 us; error 51 us; skew 500 ppm
Aug 08 04:49:50 dist-test-slave-04bb krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:50.010337 21421 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:50.011302 21421 webserver.cc:460] Webserver started at http://127.0.0.1:37939/ using document root <none> and password file <none>
I20260808 04:49:50.011909 21421 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:50.012005 21421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:50.012251 21421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:50.014382 21421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "ea2404f784734d0790e733b0d12e26b5"
format_stamp: "Formatted at 2026-08-08 04:49:50 on dist-test-slave-04bb"
server_key: "c8cb64d0b8509bb507d94146e32341b1"
server_key_iv: "b4750ded546a3b367376b4e7a4497789"
server_key_version: "encryptionkey@0"
I20260808 04:49:50.014964 21421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "ea2404f784734d0790e733b0d12e26b5"
format_stamp: "Formatted at 2026-08-08 04:49:50 on dist-test-slave-04bb"
server_key: "c8cb64d0b8509bb507d94146e32341b1"
server_key_iv: "b4750ded546a3b367376b4e7a4497789"
server_key_version: "encryptionkey@0"
I20260808 04:49:50.018685 21421 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:49:50.021344 21445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:50.022541 21421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:49:50.022692 21421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "ea2404f784734d0790e733b0d12e26b5"
format_stamp: "Formatted at 2026-08-08 04:49:50 on dist-test-slave-04bb"
server_key: "c8cb64d0b8509bb507d94146e32341b1"
server_key_iv: "b4750ded546a3b367376b4e7a4497789"
server_key_version: "encryptionkey@0"
I20260808 04:49:50.022809 21421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:50.027522 21421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:50.030670 21421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:50.030881 21421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:50.038193 21421 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37961
I20260808 04:49:50.038204 21520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37961 every 8 connection(s)
I20260808 04:49:50.039121 21421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:50.042014 21521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:50.044994 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 21421
I20260808 04:49:50.045147 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:50.045447 22538 external_mini_cluster.cc:1428] Setting key e2e14efa927ab19f2df36b6cc9096b9b
I20260808 04:49:50.048169 21521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5: Bootstrap starting.
I20260808 04:49:50.050994 21521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:50.051232 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:50.046930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:51545 (local address 127.0.0.1:37961)
0808 04:49:50.047372 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:50.047392 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:50.047436 (+ 44us) server_negotiation.cc:407] Connection header received
0808 04:49:50.048294 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:50.048319 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:50.048706 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:50.049071 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:50.049842 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:50.049847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:50.049863 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:50.049875 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:50.049942 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:50.049969 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:50.050108 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:50.050175 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 04:49:50.050330 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":116,"threads_started":1}
I20260808 04:49:50.051862 21521 log.cc:824] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:50.053915 21521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5: No bootstrap required, opened a new log
I20260808 04:49:50.054080 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:50.057977 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:50.056185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:52307 (local address 127.0.0.1:37961)
0808 04:49:50.056299 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:50.056302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:50.056588 (+ 286us) server_negotiation.cc:407] Connection header received
0808 04:49:50.056672 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:50.056674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:50.056704 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:50.056770 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:50.057569 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:50.057571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:50.057574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:50.057584 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:50.057608 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:50.057618 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:50.057688 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:50.057802 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 04:49:50.057844 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 04:49:50.058431 21521 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2404f784734d0790e733b0d12e26b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37961 } }
I20260808 04:49:50.058681 21521 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:50.058740 21521 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2404f784734d0790e733b0d12e26b5, State: Initialized, Role: FOLLOWER
I20260808 04:49:50.059212 21521 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2404f784734d0790e733b0d12e26b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37961 } }
I20260808 04:49:50.059382 21521 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:50.059474 21521 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:50.059576 21521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:50.060503 21521 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2404f784734d0790e733b0d12e26b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37961 } }
I20260808 04:49:50.060906 21521 leader_election.cc:302] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea2404f784734d0790e733b0d12e26b5; no voters:
I20260808 04:49:50.061244 21521 leader_election.cc:288] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:50.061375 21535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:50.061645 21535 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [term 1 LEADER]: Becoming Leader. State: Replica: ea2404f784734d0790e733b0d12e26b5, State: Running, Role: LEADER
I20260808 04:49:50.061997 21535 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2404f784734d0790e733b0d12e26b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37961 } }
I20260808 04:49:50.062539 21521 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:50.063642 21536 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea2404f784734d0790e733b0d12e26b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2404f784734d0790e733b0d12e26b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37961 } } }
I20260808 04:49:50.063777 21536 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:50.064150 21537 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea2404f784734d0790e733b0d12e26b5. Latest consensus state: current_term: 1 leader_uuid: "ea2404f784734d0790e733b0d12e26b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2404f784734d0790e733b0d12e26b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37961 } } }
I20260808 04:49:50.064296 21537 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:50.065016 21544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:50.068320 21544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:50.073925 21544 catalog_manager.cc:1347] Generated new cluster ID: c292b362095f43c4a1eaab0349be89b7
I20260808 04:49:50.073997 21544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:50.108073 21544 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:50.108441 21544 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:50.116421 21544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea2404f784734d0790e733b0d12e26b5: Generated new TSK 0
I20260808 04:49:50.117167 21544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:50.128929 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 21421
2026-08-08T04:49:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1626 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:50 dist-test-slave-04bb krb5kdc[21562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:50 dist-test-slave-04bb krb5kdc[21562](info): set up 2 sockets
Aug 08 04:49:50 dist-test-slave-04bb krb5kdc[21562](info): commencing operation
krb5kdc: starting...
W20260808 04:49:51.381676 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:51 dist-test-slave-04bb krb5kdc[21562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:51.507177 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:51.620645 21580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:51.620889 21580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:51.620929 21580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:51.624805 21580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:51.624850 21580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:51.624871 21580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:51.624888 21580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:51.624903 21580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:51.630033 21580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42027
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:51.630978 21580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:51.631942 21580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:51.640539 21591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:51.640517 21587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:51.640583 21589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:51.640908 21580 server_base.cc:1034] running on GCE node
I20260808 04:49:51.641379 21580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:51.642465 21580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:51.643668 21580 hybrid_clock.cc:648] HybridClock initialized: now 1786164591643658 us; error 39 us; skew 500 ppm
Aug 08 04:49:51 dist-test-slave-04bb krb5kdc[21562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164591, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:51.652448 21580 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:51.653337 21580 webserver.cc:460] Webserver started at http://127.0.0.1:38927/ using document root <none> and password file <none>
I20260808 04:49:51.653903 21580 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:51.653995 21580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:51.654237 21580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:51.656298 21580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "85172929a12d4ec98d3745abb3b7959d"
format_stamp: "Formatted at 2026-08-08 04:49:51 on dist-test-slave-04bb"
server_key: "0f0e8e399be3f1f0dee241193eb1c0da"
server_key_iv: "f83976c509e20596f99dd97c41d25d57"
server_key_version: "encryptionkey@0"
I20260808 04:49:51.656929 21580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "85172929a12d4ec98d3745abb3b7959d"
format_stamp: "Formatted at 2026-08-08 04:49:51 on dist-test-slave-04bb"
server_key: "0f0e8e399be3f1f0dee241193eb1c0da"
server_key_iv: "f83976c509e20596f99dd97c41d25d57"
server_key_version: "encryptionkey@0"
I20260808 04:49:51.660666 21580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:49:51.663130 21598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:51.664392 21580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:49:51.664599 21580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "85172929a12d4ec98d3745abb3b7959d"
format_stamp: "Formatted at 2026-08-08 04:49:51 on dist-test-slave-04bb"
server_key: "0f0e8e399be3f1f0dee241193eb1c0da"
server_key_iv: "f83976c509e20596f99dd97c41d25d57"
server_key_version: "encryptionkey@0"
I20260808 04:49:51.664745 21580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:51.677232 21580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:51.680406 21580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:51.680675 21580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:51.688571 21580 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42027
I20260808 04:49:51.688844 21676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42027 every 8 connection(s)
I20260808 04:49:51.689572 21580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:51.692339 21679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:51.693352 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 21580
I20260808 04:49:51.693500 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:51.693789 22538 external_mini_cluster.cc:1428] Setting key 2524a413b1c9dbdaf4c86b33149beaf0
I20260808 04:49:51.699220 21679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d: Bootstrap starting.
I20260808 04:49:51.701807 21679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:51.702539 21679 log.cc:824] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:49:51 dist-test-slave-04bb krb5kdc[21562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 04:49:51.705072 21679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d: No bootstrap required, opened a new log
I20260808 04:49:51.709441 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:51.695309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:44829 (local address 127.0.0.1:42027)
0808 04:49:51.695746 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:51.695768 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:51.695823 (+ 55us) server_negotiation.cc:407] Connection header received
0808 04:49:51.696670 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:51.696694 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:51.697142 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:51.697535 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:51.698248 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:51.699677 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:51.700553 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:51.701223 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:51.701386 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:51.704454 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:51.704460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:51.704472 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:51.704499 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:51.706445 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:51.707036 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:51.707041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:51.707045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:51.707155 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:51.707566 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:51.707569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:51.707570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:51.707950 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:51.708107 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:51.708472 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 04:49:51.708721 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":108,"threads_started":1}
I20260808 04:49:51.709501 21679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85172929a12d4ec98d3745abb3b7959d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42027 } }
I20260808 04:49:51.709824 21679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:51.709908 21679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85172929a12d4ec98d3745abb3b7959d, State: Initialized, Role: FOLLOWER
I20260808 04:49:51.710399 21679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85172929a12d4ec98d3745abb3b7959d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42027 } }
I20260808 04:49:51.710556 21679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:51.710628 21679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:51.710731 21679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:51.711764 21679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85172929a12d4ec98d3745abb3b7959d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42027 } }
I20260808 04:49:51.712118 21679 leader_election.cc:302] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85172929a12d4ec98d3745abb3b7959d; no voters:
I20260808 04:49:51.712442 21679 leader_election.cc:288] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:51.712687 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:51.712718 21692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:51.712975 21692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [term 1 LEADER]: Becoming Leader. State: Replica: 85172929a12d4ec98d3745abb3b7959d, State: Running, Role: LEADER
I20260808 04:49:51.713238 21692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85172929a12d4ec98d3745abb3b7959d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42027 } }
I20260808 04:49:51.713869 21679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:51.714371 21692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85172929a12d4ec98d3745abb3b7959d. Latest consensus state: current_term: 1 leader_uuid: "85172929a12d4ec98d3745abb3b7959d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85172929a12d4ec98d3745abb3b7959d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42027 } } }
I20260808 04:49:51.714486 21692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:51.714841 21694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85172929a12d4ec98d3745abb3b7959d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85172929a12d4ec98d3745abb3b7959d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42027 } } }
I20260808 04:49:51.714931 21694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:51.714934 21704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:51.717926 21704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:51.723291 21704 catalog_manager.cc:1347] Generated new cluster ID: 637102df1a954f8a819f7bec1da081fa
I20260808 04:49:51.723349 21704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:51.730068 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:51.715136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:41891 (local address 127.0.0.1:42027)
0808 04:49:51.715238 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:51.715242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:51.720117 (+ 4875us) server_negotiation.cc:407] Connection header received
0808 04:49:51.720220 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:51.720224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:51.720273 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:51.720365 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:51.721361 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:51.722248 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:51.723483 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:51.723856 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:51.723935 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:51.725854 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:51.725858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:51.725861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:51.725888 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:51.727775 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:51.728409 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:51.728415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:51.728419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:51.728508 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:51.728963 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:51.728966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:51.728968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:51.729188 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:51.729336 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:51.729766 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 04:49:51.729897 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":39}
I20260808 04:49:51.738200 21704 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:51.739362 21704 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:51.749818 21704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85172929a12d4ec98d3745abb3b7959d: Generated new TSK 0
I20260808 04:49:51.750490 21704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:51.756008 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 21580
W20260808 04:49:51.768441 21560 connection.cc:535] client connection to 127.0.0.1:42027 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42027): Connection reset by peer (error 104)
2026-08-08T04:49:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1621 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:51 dist-test-slave-04bb krb5kdc[21725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:51 dist-test-slave-04bb krb5kdc[21725](info): set up 2 sockets
Aug 08 04:49:51 dist-test-slave-04bb krb5kdc[21725](info): commencing operation
krb5kdc: starting...
W20260808 04:49:53.027019 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:53 dist-test-slave-04bb krb5kdc[21725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:53.155006 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43095
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:53.267727 21739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:53.268049 21739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:53.268126 21739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:53.272100 21739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:53.272192 21739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:53.272287 21739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:53.272330 21739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:53.272392 21739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:53.277489 21739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43095
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:53.278505 21739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:53.279520 21739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:53.288120 21749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:53.288237 21739 server_base.cc:1034] running on GCE node
W20260808 04:49:53.288162 21748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:53.288120 21751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:53.288957 21739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:53.290024 21739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:53.291222 21739 hybrid_clock.cc:648] HybridClock initialized: now 1786164593291205 us; error 28 us; skew 500 ppm
Aug 08 04:49:53 dist-test-slave-04bb krb5kdc[21725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164593, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:53.294113 21739 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:53.294947 21739 webserver.cc:460] Webserver started at http://127.0.0.1:40611/ using document root <none> and password file <none>
I20260808 04:49:53.295466 21739 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:53.295526 21739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:53.295694 21739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:53.297811 21739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "e7514384e0c942ce90f7a15986dffa2a"
format_stamp: "Formatted at 2026-08-08 04:49:53 on dist-test-slave-04bb"
server_key: "fe1aceaedcebaf913603d71a9d711a33"
server_key_iv: "32de0a05107e7358fbf495ae427d3cac"
server_key_version: "encryptionkey@0"
I20260808 04:49:53.298350 21739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "e7514384e0c942ce90f7a15986dffa2a"
format_stamp: "Formatted at 2026-08-08 04:49:53 on dist-test-slave-04bb"
server_key: "fe1aceaedcebaf913603d71a9d711a33"
server_key_iv: "32de0a05107e7358fbf495ae427d3cac"
server_key_version: "encryptionkey@0"
I20260808 04:49:53.302078 21739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:49:53.304497 21764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:53.305740 21739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:49:53.305861 21739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "e7514384e0c942ce90f7a15986dffa2a"
format_stamp: "Formatted at 2026-08-08 04:49:53 on dist-test-slave-04bb"
server_key: "fe1aceaedcebaf913603d71a9d711a33"
server_key_iv: "32de0a05107e7358fbf495ae427d3cac"
server_key_version: "encryptionkey@0"
I20260808 04:49:53.305946 21739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:53.317342 21739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:53.320035 21739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:53.320186 21739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:53.327138 21739 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43095
I20260808 04:49:53.327150 21843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43095 every 8 connection(s)
I20260808 04:49:53.328091 21739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:53.330972 21844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:53.331107 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 21739
I20260808 04:49:53.331230 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:53.331557 22538 external_mini_cluster.cc:1428] Setting key d430e484f6c185bb1c29fd30b75b3019
I20260808 04:49:53.337353 21844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a: Bootstrap starting.
I20260808 04:49:53.339620 21844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:53.340430 21844 log.cc:824] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:53.340739 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:53.333014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:49501 (local address 127.0.0.1:43095)
0808 04:49:53.333455 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:53.333469 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:53.333504 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:49:53.334283 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:53.334301 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:53.334642 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:53.334980 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:53.335378 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:53.336328 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:53.337306 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:53.337818 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:53.337998 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:53.338924 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:53.338929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:53.338941 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:53.339339 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:53.339392 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:53.339416 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:53.339562 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:53.339754 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 04:49:53.339894 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":126,"threads_started":1}
I20260808 04:49:53.342590 21844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a: No bootstrap required, opened a new log
I20260808 04:49:53.343776 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:53.346805 21844 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7514384e0c942ce90f7a15986dffa2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } }
I20260808 04:49:53.347066 21844 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:53.347113 21844 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7514384e0c942ce90f7a15986dffa2a, State: Initialized, Role: FOLLOWER
I20260808 04:49:53.347571 21844 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7514384e0c942ce90f7a15986dffa2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } }
I20260808 04:49:53.347743 21844 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:53.347815 21844 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:53.347930 21844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:53.349099 21844 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7514384e0c942ce90f7a15986dffa2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } }
I20260808 04:49:53.349517 21844 leader_election.cc:302] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7514384e0c942ce90f7a15986dffa2a; no voters:
I20260808 04:49:53.349857 21844 leader_election.cc:288] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:53.349956 21859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:53.350196 21859 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [term 1 LEADER]: Becoming Leader. State: Replica: e7514384e0c942ce90f7a15986dffa2a, State: Running, Role: LEADER
I20260808 04:49:53.350392 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:53.345841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:56751 (local address 127.0.0.1:43095)
0808 04:49:53.345970 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:53.345975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:53.346114 (+ 139us) server_negotiation.cc:407] Connection header received
0808 04:49:53.346208 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:53.346211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:53.346259 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:53.346338 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:53.346586 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:53.347309 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:53.348443 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:53.348742 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:53.348829 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:53.349741 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:53.349744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:53.349746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:53.349757 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:53.349797 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:53.349813 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:53.349910 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:53.350241 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 04:49:53.350274 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:49:53.350661 21859 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7514384e0c942ce90f7a15986dffa2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } }
I20260808 04:49:53.351182 21844 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:53.351850 21859 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7514384e0c942ce90f7a15986dffa2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7514384e0c942ce90f7a15986dffa2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } } }
I20260808 04:49:53.352042 21859 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:53.352316 21860 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7514384e0c942ce90f7a15986dffa2a. Latest consensus state: current_term: 1 leader_uuid: "e7514384e0c942ce90f7a15986dffa2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7514384e0c942ce90f7a15986dffa2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43095 } } }
I20260808 04:49:53.352466 21860 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:53.352551 21872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:53.355564 21872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:53.361399 21872 catalog_manager.cc:1347] Generated new cluster ID: 937f7e6db9c143b7b7b9d09b2ba402a6
I20260808 04:49:53.361476 21872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:53.368148 21872 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:53.369380 21872 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:53.378996 21872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7514384e0c942ce90f7a15986dffa2a: Generated new TSK 0
I20260808 04:49:53.379761 21872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:53.384259 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 21739
W20260808 04:49:53.395264 21723 connection.cc:535] client connection to 127.0.0.1:43095 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43095): Connection reset by peer (error 104)
2026-08-08T04:49:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1626 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:53 dist-test-slave-04bb krb5kdc[21887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:53 dist-test-slave-04bb krb5kdc[21887](info): set up 2 sockets
Aug 08 04:49:53 dist-test-slave-04bb krb5kdc[21887](info): commencing operation
krb5kdc: starting...
W20260808 04:49:54.630964 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:54 dist-test-slave-04bb krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 04:49:54.755745 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:54.869027 21905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:54.869328 21905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:54.869407 21905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:54.873396 21905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:54.873477 21905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:54.873517 21905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:54.873574 21905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:54.873610 21905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:54.878728 21905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:54.879828 21905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:54.880947 21905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:54.890305 21912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:54.890355 21915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:54.890421 21905 server_base.cc:1034] running on GCE node
W20260808 04:49:54.890590 21911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:54.891196 21905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:54.892328 21905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:54.893527 21905 hybrid_clock.cc:648] HybridClock initialized: now 1786164594893531 us; error 47 us; skew 500 ppm
Aug 08 04:49:54 dist-test-slave-04bb krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:54.896781 21905 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 04:49:54.899231 21905 webserver.cc:460] Webserver started at http://127.0.0.1:46731/ using document root <none> and password file <none>
I20260808 04:49:54.899863 21905 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:54.899933 21905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:54.900204 21905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:54.902499 21905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "7ccf2deec60d4245b21ef47a7a153f7f"
format_stamp: "Formatted at 2026-08-08 04:49:54 on dist-test-slave-04bb"
server_key: "70ad3a24421f996787e6529b0a346b6b"
server_key_iv: "f79b0870ff111fbd6aafecb8d3d0c059"
server_key_version: "encryptionkey@0"
I20260808 04:49:54.903127 21905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "7ccf2deec60d4245b21ef47a7a153f7f"
format_stamp: "Formatted at 2026-08-08 04:49:54 on dist-test-slave-04bb"
server_key: "70ad3a24421f996787e6529b0a346b6b"
server_key_iv: "f79b0870ff111fbd6aafecb8d3d0c059"
server_key_version: "encryptionkey@0"
I20260808 04:49:54.907326 21905 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 04:49:54.909909 21924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:54.911103 21905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 04:49:54.911235 21905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "7ccf2deec60d4245b21ef47a7a153f7f"
format_stamp: "Formatted at 2026-08-08 04:49:54 on dist-test-slave-04bb"
server_key: "70ad3a24421f996787e6529b0a346b6b"
server_key_iv: "f79b0870ff111fbd6aafecb8d3d0c059"
server_key_version: "encryptionkey@0"
I20260808 04:49:54.911339 21905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:54.915951 21905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:54.919252 21905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:54.919452 21905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:54.926463 21905 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44239
I20260808 04:49:54.926493 21995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44239 every 8 connection(s)
I20260808 04:49:54.927380 21905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:54.930172 21996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:54.931734 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 21905
I20260808 04:49:54.931890 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:54.932214 22538 external_mini_cluster.cc:1428] Setting key 5a87100e6835b34dadcc78b1201e4141
I20260808 04:49:54.936718 21996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f: Bootstrap starting.
I20260808 04:49:54.937589 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:54.933665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:33079 (local address 127.0.0.1:44239)
0808 04:49:54.934103 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:54.934117 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:54.934152 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:49:54.934872 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:54.934890 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:54.935250 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:54.935593 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:54.936384 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:54.936389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:54.936401 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:54.936412 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:54.936469 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:54.936488 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:54.936689 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:54.936765 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 04:49:54.936929 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":135,"threads_started":1}
I20260808 04:49:54.939003 21996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:54.939882 21996 log.cc:824] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:54.940348 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:54.941931 21996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f: No bootstrap required, opened a new log
I20260808 04:49:54.945825 21996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccf2deec60d4245b21ef47a7a153f7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } }
I20260808 04:49:54.946067 21996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:54.946133 21996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ccf2deec60d4245b21ef47a7a153f7f, State: Initialized, Role: FOLLOWER
I20260808 04:49:54.946259 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:54.942485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:54795 (local address 127.0.0.1:44239)
0808 04:49:54.942600 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:54.942603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:54.944816 (+ 2213us) server_negotiation.cc:407] Connection header received
0808 04:49:54.944978 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:54.944983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:54.945019 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:54.945104 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:54.945829 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:54.945833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:54.945836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:54.945848 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:54.945875 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:54.945894 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:54.945985 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:54.946061 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 04:49:54.946102 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:49:54.946623 21996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccf2deec60d4245b21ef47a7a153f7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } }
I20260808 04:49:54.946750 21996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:54.946841 21996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:54.946954 21996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:54.947926 21996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccf2deec60d4245b21ef47a7a153f7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } }
I20260808 04:49:54.948266 21996 leader_election.cc:302] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ccf2deec60d4245b21ef47a7a153f7f; no voters:
I20260808 04:49:54.948612 21996 leader_election.cc:288] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:54.948696 22010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:54.948944 22010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [term 1 LEADER]: Becoming Leader. State: Replica: 7ccf2deec60d4245b21ef47a7a153f7f, State: Running, Role: LEADER
I20260808 04:49:54.949299 22010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccf2deec60d4245b21ef47a7a153f7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } }
I20260808 04:49:54.949883 21996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:54.950963 22011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ccf2deec60d4245b21ef47a7a153f7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccf2deec60d4245b21ef47a7a153f7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } } }
I20260808 04:49:54.951092 22011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:54.951437 22012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ccf2deec60d4245b21ef47a7a153f7f. Latest consensus state: current_term: 1 leader_uuid: "7ccf2deec60d4245b21ef47a7a153f7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccf2deec60d4245b21ef47a7a153f7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } } }
I20260808 04:49:54.951608 22012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:54.952201 22022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:54.955029 22022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:54.960059 22022 catalog_manager.cc:1347] Generated new cluster ID: 0679f30d8557459ca45f5fcd01a5f8a6
I20260808 04:49:54.960122 22022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:54.973443 22022 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:54.973759 22022 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:54.984273 22022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ccf2deec60d4245b21ef47a7a153f7f: Generated new TSK 0
I20260808 04:49:54.985000 22022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:54.992905 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 21905
2026-08-08T04:49:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1611 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:55 dist-test-slave-04bb krb5kdc[22040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:55 dist-test-slave-04bb krb5kdc[22040](info): set up 2 sockets
Aug 08 04:49:55 dist-test-slave-04bb krb5kdc[22040](info): commencing operation
krb5kdc: starting...
W20260808 04:49:56.324926 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:56 dist-test-slave-04bb krb5kdc[22040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:49:56.450923 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:39317
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:39317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:56.562363 22061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:56.562662 22061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:56.562739 22061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:56.566713 22061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:56.566813 22061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:56.566852 22061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:56.566915 22061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:56.566955 22061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:56.572149 22061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:39317
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:39317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:56.573252 22061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:56.574306 22061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:56.582924 22074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:56.582986 22068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:56.582986 22069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:56.583102 22061 server_base.cc:1034] running on GCE node
I20260808 04:49:56.583877 22061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:56.585013 22061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:56.586179 22061 hybrid_clock.cc:648] HybridClock initialized: now 1786164596586152 us; error 38 us; skew 500 ppm
Aug 08 04:49:56 dist-test-slave-04bb krb5kdc[22040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164596, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:56.589102 22061 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:49:56.590020 22061 webserver.cc:460] Webserver started at http://127.22.2.190:42115/ using document root <none> and password file <none>
I20260808 04:49:56.590590 22061 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:56.590682 22061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:56.590922 22061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:56.593026 22061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "c555d4eb010c44a1a340d7666253749f"
format_stamp: "Formatted at 2026-08-08 04:49:56 on dist-test-slave-04bb"
server_key: "50451fc32c6693ba7fa4c875036ae515"
server_key_iv: "4abd6cdd2e977c0f8965171be35286bf"
server_key_version: "encryptionkey@0"
I20260808 04:49:56.593605 22061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "c555d4eb010c44a1a340d7666253749f"
format_stamp: "Formatted at 2026-08-08 04:49:56 on dist-test-slave-04bb"
server_key: "50451fc32c6693ba7fa4c875036ae515"
server_key_iv: "4abd6cdd2e977c0f8965171be35286bf"
server_key_version: "encryptionkey@0"
I20260808 04:49:56.597340 22061 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 04:49:56.599722 22083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:56.601006 22061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:49:56.601142 22061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "c555d4eb010c44a1a340d7666253749f"
format_stamp: "Formatted at 2026-08-08 04:49:56 on dist-test-slave-04bb"
server_key: "50451fc32c6693ba7fa4c875036ae515"
server_key_iv: "4abd6cdd2e977c0f8965171be35286bf"
server_key_version: "encryptionkey@0"
I20260808 04:49:56.601251 22061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:56.614745 22061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:56.617830 22061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:56.618026 22061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:56.625281 22061 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:39317
I20260808 04:49:56.625315 22158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:39317 every 8 connection(s)
I20260808 04:49:56.626187 22061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:56.626976 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22061
I20260808 04:49:56.627123 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:56.627461 22538 external_mini_cluster.cc:1428] Setting key 7a6f35e9064cb990558ee25f2940cf3f
I20260808 04:49:56.629716 22160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:56.635659 22160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f: Bootstrap starting.
Aug 08 04:49:56 dist-test-slave-04bb krb5kdc[22040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:49:56.638569 22160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:56.639402 22160 log.cc:824] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:56.641551 22160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f: No bootstrap required, opened a new log
I20260808 04:49:56.641923 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:56.628978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44825 (local address 127.22.2.190:39317)
0808 04:49:56.629422 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:56.629437 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:56.629472 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:49:56.630142 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:56.630160 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:56.630483 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:56.630828 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:56.631673 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:56.632719 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:56.633611 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:56.634074 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:56.634242 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:56.637178 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:56.637183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:56.637196 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:56.637221 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:56.639055 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:56.639569 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:56.639575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:56.639579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:56.639685 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:56.640036 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:56.640039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:56.640041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:56.640417 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:56.640665 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:56.640985 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 04:49:56.641195 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":146,"threads_started":1}
I20260808 04:49:56.645419 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:56.645691 22160 raft_consensus.cc:348] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555d4eb010c44a1a340d7666253749f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39317 } }
I20260808 04:49:56.645937 22160 raft_consensus.cc:374] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:56.646008 22160 raft_consensus.cc:729] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c555d4eb010c44a1a340d7666253749f, State: Initialized, Role: FOLLOWER
I20260808 04:49:56.646454 22160 consensus_queue.cc:260] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555d4eb010c44a1a340d7666253749f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39317 } }
I20260808 04:49:56.646646 22160 raft_consensus.cc:388] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:56.646754 22160 raft_consensus.cc:482] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:56.646888 22160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:56.648051 22160 raft_consensus.cc:504] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555d4eb010c44a1a340d7666253749f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39317 } }
I20260808 04:49:56.648443 22160 leader_election.cc:302] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c555d4eb010c44a1a340d7666253749f; no voters:
I20260808 04:49:56.648833 22160 leader_election.cc:288] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:56.648942 22177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:56.649156 22177 raft_consensus.cc:686] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [term 1 LEADER]: Becoming Leader. State: Replica: c555d4eb010c44a1a340d7666253749f, State: Running, Role: LEADER
I20260808 04:49:56.649490 22177 consensus_queue.cc:237] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555d4eb010c44a1a340d7666253749f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39317 } }
I20260808 04:49:56.650189 22160 sys_catalog.cc:564] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:56.650619 22177 sys_catalog.cc:455] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c555d4eb010c44a1a340d7666253749f. Latest consensus state: current_term: 1 leader_uuid: "c555d4eb010c44a1a340d7666253749f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555d4eb010c44a1a340d7666253749f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39317 } } }
I20260808 04:49:56.650825 22177 sys_catalog.cc:458] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:56.651095 22178 sys_catalog.cc:455] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c555d4eb010c44a1a340d7666253749f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c555d4eb010c44a1a340d7666253749f" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 39317 } } }
I20260808 04:49:56.651194 22178 sys_catalog.cc:458] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:56.651381 22185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:56.654230 22185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:56.659456 22185 catalog_manager.cc:1347] Generated new cluster ID: 3e3fd4de2da14634983c1b66dbb27fa0
I20260808 04:49:56.659514 22185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:56.662236 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:56.647488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38143 (local address 127.22.2.190:39317)
0808 04:49:56.647620 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:56.647624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:56.647745 (+ 121us) server_negotiation.cc:407] Connection header received
0808 04:49:56.647818 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:56.647836 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:56.647892 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:56.647987 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:56.648758 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:56.649587 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:56.650857 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:56.651282 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:56.651395 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:56.657560 (+ 6165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:56.657565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:56.657569 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:49:56.657596 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:56.659496 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:56.660266 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:56.660270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:56.660272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:56.660362 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:49:56.660855 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:49:56.660859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:49:56.660861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:49:56.661088 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:56.661242 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:56.661961 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 04:49:56.662065 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 04:49:56.668208 22185 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:56.669234 22185 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:56.674324 22185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c555d4eb010c44a1a340d7666253749f: Generated new TSK 0
I20260808 04:49:56.674979 22185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:56.683905 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22061
W20260808 04:49:56.693168 22037 connection.cc:535] client connection to 127.22.2.190:39317 recv error: Network error: failed to read from TLS socket (remote: 127.22.2.190:39317): Connection reset by peer (error 104)
2026-08-08T04:49:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1686 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:56 dist-test-slave-04bb krb5kdc[22202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:56 dist-test-slave-04bb krb5kdc[22202](info): set up 2 sockets
Aug 08 04:49:56 dist-test-slave-04bb krb5kdc[22202](info): commencing operation
krb5kdc: starting...
W20260808 04:49:57.911446 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:58 dist-test-slave-04bb krb5kdc[22202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:49:58.044032 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33301
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:33301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:58.162370 22217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:58.162662 22217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:58.162752 22217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:58.166779 22217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:58.166868 22217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:58.166906 22217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:58.166965 22217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:58.167003 22217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:58.172384 22217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:33301
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:58.173585 22217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:58.174675 22217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:58.183475 22228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:58.183525 22225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:58.183473 22224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:58.183822 22217 server_base.cc:1034] running on GCE node
I20260808 04:49:58.184353 22217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:58.185526 22217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:58.186688 22217 hybrid_clock.cc:648] HybridClock initialized: now 1786164598186670 us; error 31 us; skew 500 ppm
Aug 08 04:49:58 dist-test-slave-04bb krb5kdc[22202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164598, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:58.189466 22217 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:49:58.190337 22217 webserver.cc:460] Webserver started at http://127.22.2.190:33237/ using document root <none> and password file <none>
I20260808 04:49:58.190904 22217 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:58.190968 22217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:58.191224 22217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:58.193397 22217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "9dc955fc902c408fa8ea5190366aafe7"
format_stamp: "Formatted at 2026-08-08 04:49:58 on dist-test-slave-04bb"
server_key: "2c5ac5c1f7e34aae3e9f86439a2adf9b"
server_key_iv: "8def54084fff431cf5035c357a464a20"
server_key_version: "encryptionkey@0"
I20260808 04:49:58.193966 22217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "9dc955fc902c408fa8ea5190366aafe7"
format_stamp: "Formatted at 2026-08-08 04:49:58 on dist-test-slave-04bb"
server_key: "2c5ac5c1f7e34aae3e9f86439a2adf9b"
server_key_iv: "8def54084fff431cf5035c357a464a20"
server_key_version: "encryptionkey@0"
I20260808 04:49:58.197796 22217 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:49:58.200237 22242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:58.201555 22217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:49:58.201704 22217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "9dc955fc902c408fa8ea5190366aafe7"
format_stamp: "Formatted at 2026-08-08 04:49:58 on dist-test-slave-04bb"
server_key: "2c5ac5c1f7e34aae3e9f86439a2adf9b"
server_key_iv: "8def54084fff431cf5035c357a464a20"
server_key_version: "encryptionkey@0"
I20260808 04:49:58.201830 22217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:58.222357 22217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:58.226159 22217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:58.226383 22217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:58.234227 22217 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:33301
I20260808 04:49:58.234306 22316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:33301 every 8 connection(s)
I20260808 04:49:58.235240 22217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:58.238368 22318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:58.240142 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22217
I20260808 04:49:58.240319 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:58.240669 22538 external_mini_cluster.cc:1428] Setting key 0670efebddc9608414b5ac69b000f5b1
I20260808 04:49:58.246223 22318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7: Bootstrap starting.
I20260808 04:49:58.249157 22318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:58.250208 22318 log.cc:824] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:58.251749 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:58.242159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40057 (local address 127.22.2.190:33301)
0808 04:49:58.242632 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:58.242655 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:58.242705 (+ 50us) server_negotiation.cc:407] Connection header received
0808 04:49:58.243645 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:58.243674 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:58.244174 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:58.244614 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:58.245122 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:58.246731 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:58.247635 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:58.248422 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:58.248659 (+ 237us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:58.249593 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:58.249601 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:58.249619 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:58.250195 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:58.250275 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:58.250315 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:58.250484 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:58.250690 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 04:49:58.250845 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":103,"threads_started":1}
I20260808 04:49:58.252565 22318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7: No bootstrap required, opened a new log
I20260808 04:49:58.254918 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:58.257123 22318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc955fc902c408fa8ea5190366aafe7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33301 } }
I20260808 04:49:58.257362 22318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:58.257441 22318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dc955fc902c408fa8ea5190366aafe7, State: Initialized, Role: FOLLOWER
I20260808 04:49:58.257900 22318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc955fc902c408fa8ea5190366aafe7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33301 } }
I20260808 04:49:58.258054 22318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:58.258126 22318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:58.258224 22318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:58.259135 22318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc955fc902c408fa8ea5190366aafe7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33301 } }
I20260808 04:49:58.259534 22318 leader_election.cc:302] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dc955fc902c408fa8ea5190366aafe7; no voters:
I20260808 04:49:58.259928 22318 leader_election.cc:288] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:58.260061 22333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:58.260351 22333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [term 1 LEADER]: Becoming Leader. State: Replica: 9dc955fc902c408fa8ea5190366aafe7, State: Running, Role: LEADER
I20260808 04:49:58.260819 22333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc955fc902c408fa8ea5190366aafe7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33301 } }
I20260808 04:49:58.261276 22318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:58.262820 22334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dc955fc902c408fa8ea5190366aafe7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc955fc902c408fa8ea5190366aafe7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33301 } } }
I20260808 04:49:58.262954 22334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:58.263204 22335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dc955fc902c408fa8ea5190366aafe7. Latest consensus state: current_term: 1 leader_uuid: "9dc955fc902c408fa8ea5190366aafe7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dc955fc902c408fa8ea5190366aafe7" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33301 } } }
I20260808 04:49:58.263307 22335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7 [sys.catalog]: This master's current role is: LEADER
W20260808 04:49:58.267699 22355 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:49:58.268986 22355 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:49:58.269526 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:58.264580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53891 (local address 127.22.2.190:33301)
0808 04:49:58.264706 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:58.264712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:58.264789 (+ 77us) server_negotiation.cc:407] Connection header received
0808 04:49:58.264997 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:58.265000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:58.265051 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:58.265109 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:58.265552 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:58.266299 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:58.267458 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:49:58.267785 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:49:58.267867 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:49:58.268996 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:58.269000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:58.269003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:58.269015 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:58.269041 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:58.269053 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:58.269106 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:58.269349 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 04:49:58.269387 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 04:49:58.275663 22352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:58.277081 22352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:58.285107 22352 catalog_manager.cc:1347] Generated new cluster ID: a98104929074417f97ba3744503bb9dc
I20260808 04:49:58.285233 22352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:58.298461 22352 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:58.299546 22352 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:58.306449 22352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dc955fc902c408fa8ea5190366aafe7: Generated new TSK 0
I20260808 04:49:58.307147 22352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:58.312268 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22217
W20260808 04:49:58.322947 22200 connection.cc:535] client connection to 127.22.2.190:33301 recv error: Network error: failed to read from TLS socket (remote: 127.22.2.190:33301): Connection reset by peer (error 104)
2026-08-08T04:49:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1629 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:58 dist-test-slave-04bb krb5kdc[22364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:58 dist-test-slave-04bb krb5kdc[22364](info): set up 2 sockets
Aug 08 04:49:58 dist-test-slave-04bb krb5kdc[22364](info): commencing operation
krb5kdc: starting...
W20260808 04:49:59.544739 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:49:59 dist-test-slave-04bb krb5kdc[22364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:49:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:49:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:49:59.671226 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38161
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:49:59.786433 22381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:49:59.786760 22381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:49:59.786837 22381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:49:59.790912 22381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:49:59.791007 22381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:49:59.791047 22381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:49:59.791101 22381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:49:59.791141 22381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:49:59.796419 22381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:49:59.797602 22381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:49:59.798686 22381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:49:59.807863 22389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:49:59.807837 22391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:59.808207 22381 server_base.cc:1034] running on GCE node
W20260808 04:49:59.807855 22393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:49:59.808975 22381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:49:59.810277 22381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:49:59.811444 22381 hybrid_clock.cc:648] HybridClock initialized: now 1786164599811421 us; error 43 us; skew 500 ppm
Aug 08 04:49:59 dist-test-slave-04bb krb5kdc[22364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164599, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:49:59.814373 22381 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:49:59.815311 22381 webserver.cc:460] Webserver started at http://127.22.2.190:44249/ using document root <none> and password file <none>
I20260808 04:49:59.815896 22381 fs_manager.cc:359] Metadata directory not provided
I20260808 04:49:59.815987 22381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:49:59.816227 22381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:49:59.818349 22381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "918d0638d508479db471e63b943737eb"
format_stamp: "Formatted at 2026-08-08 04:49:59 on dist-test-slave-04bb"
server_key: "0b68fefe1ae820e29ade5f54adc8c916"
server_key_iv: "21547312c11fb56a3e7aaec83914ebd7"
server_key_version: "encryptionkey@0"
I20260808 04:49:59.818923 22381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "918d0638d508479db471e63b943737eb"
format_stamp: "Formatted at 2026-08-08 04:49:59 on dist-test-slave-04bb"
server_key: "0b68fefe1ae820e29ade5f54adc8c916"
server_key_iv: "21547312c11fb56a3e7aaec83914ebd7"
server_key_version: "encryptionkey@0"
I20260808 04:49:59.822710 22381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 04:49:59.825294 22401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:49:59.826614 22381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:49:59.826759 22381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "918d0638d508479db471e63b943737eb"
format_stamp: "Formatted at 2026-08-08 04:49:59 on dist-test-slave-04bb"
server_key: "0b68fefe1ae820e29ade5f54adc8c916"
server_key_iv: "21547312c11fb56a3e7aaec83914ebd7"
server_key_version: "encryptionkey@0"
I20260808 04:49:59.826880 22381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:49:59.831584 22381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:49:59.834978 22381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:49:59.835173 22381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:49:59.842314 22381 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:38161
I20260808 04:49:59.842345 22486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:38161 every 8 connection(s)
I20260808 04:49:59.843236 22381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:49:59.846141 22487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:49:59.847450 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22381
I20260808 04:49:59.847601 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:49:59.847941 22538 external_mini_cluster.cc:1428] Setting key 2142d4d430c20ac8b0f4757e87e2e33c
I20260808 04:49:59.852784 22487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb: Bootstrap starting.
I20260808 04:49:59.853370 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:59.849420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47319 (local address 127.22.2.190:38161)
0808 04:49:59.849868 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:59.849883 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:59.849920 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:49:59.850534 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:59.850551 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:59.850827 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:59.851162 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:59.851986 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:59.851991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:59.852004 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:59.852015 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:59.852069 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:59.852089 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:59.852244 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:59.852327 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 04:49:59.852486 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":134,"threads_started":1}
I20260808 04:49:59.855374 22487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb: Neither blocks nor log segments found. Creating new log.
I20260808 04:49:59.856374 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:49:59.856343 22487 log.cc:824] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb: Log is configured to *not* fsync() on all Append() calls
I20260808 04:49:59.858408 22487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb: No bootstrap required, opened a new log
I20260808 04:49:59.860188 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:49:59.858466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37677 (local address 127.22.2.190:38161)
0808 04:49:59.858582 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 04:49:59.858587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:49:59.858816 (+ 229us) server_negotiation.cc:407] Connection header received
0808 04:49:59.858958 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:49:59.858962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:49:59.859003 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:49:59.859085 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:49:59.859808 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:49:59.859812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:49:59.859815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:49:59.859827 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:49:59.859856 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:49:59.859867 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:49:59.859961 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 04:49:59.860041 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 04:49:59.860080 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:49:59.862680 22487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918d0638d508479db471e63b943737eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38161 } }
I20260808 04:49:59.863015 22487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:49:59.863119 22487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918d0638d508479db471e63b943737eb, State: Initialized, Role: FOLLOWER
I20260808 04:49:59.863639 22487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918d0638d508479db471e63b943737eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38161 } }
I20260808 04:49:59.863793 22487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:49:59.863867 22487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:49:59.863965 22487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:49:59.864959 22487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918d0638d508479db471e63b943737eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38161 } }
I20260808 04:49:59.865320 22487 leader_election.cc:302] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 918d0638d508479db471e63b943737eb; no voters:
I20260808 04:49:59.865623 22487 leader_election.cc:288] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:49:59.865746 22503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:49:59.865996 22503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [term 1 LEADER]: Becoming Leader. State: Replica: 918d0638d508479db471e63b943737eb, State: Running, Role: LEADER
I20260808 04:49:59.866348 22503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918d0638d508479db471e63b943737eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38161 } }
I20260808 04:49:59.866941 22487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:49:59.868175 22507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 918d0638d508479db471e63b943737eb. Latest consensus state: current_term: 1 leader_uuid: "918d0638d508479db471e63b943737eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918d0638d508479db471e63b943737eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38161 } } }
I20260808 04:49:59.868323 22507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:59.868744 22520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:49:59.868708 22504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "918d0638d508479db471e63b943737eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918d0638d508479db471e63b943737eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38161 } } }
I20260808 04:49:59.869201 22504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb [sys.catalog]: This master's current role is: LEADER
I20260808 04:49:59.872026 22520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:49:59.878188 22520 catalog_manager.cc:1347] Generated new cluster ID: a81385684ed544a88b5c23324d557658
I20260808 04:49:59.878274 22520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:49:59.910466 22520 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:49:59.910825 22520 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:49:59.918731 22520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 918d0638d508479db471e63b943737eb: Generated new TSK 0
I20260808 04:49:59.919507 22520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:49:59.924893 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22381
2026-08-08T04:49:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:49:59 dist-test-slave-04bb krb5kdc[22536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:49:59 dist-test-slave-04bb krb5kdc[22536](info): set up 2 sockets
Aug 08 04:49:59 dist-test-slave-04bb krb5kdc[22536](info): commencing operation
krb5kdc: starting...
W20260808 04:50:01.181205 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:50:01 dist-test-slave-04bb krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:50:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:50:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:50:01.308295 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37945
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:41435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:37945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:50:01.421243 22558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:01.421531 22558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:01.421604 22558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:50:01.425542 22558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:01.425626 22558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:01.425664 22558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:01.425699 22558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:01.425731 22558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:01.431452 22558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:37945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:01.432502 22558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:01.433560 22558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:01.442003 22565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:01.441998 22564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:01.441998 22567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:01.442330 22558 server_base.cc:1034] running on GCE node
I20260808 04:50:01.442902 22558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:01.443957 22558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:01.445168 22558 hybrid_clock.cc:648] HybridClock initialized: now 1786164601445148 us; error 31 us; skew 500 ppm
Aug 08 04:50:01 dist-test-slave-04bb krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:01.447942 22558 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:50:01.448865 22558 webserver.cc:460] Webserver started at http://127.22.2.190:37139/ using document root <none> and password file <none>
I20260808 04:50:01.449533 22558 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:01.449636 22558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:01.449872 22558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:01.452068 22558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "ebcae3e621cc472ebf1bc8243a252cb9"
format_stamp: "Formatted at 2026-08-08 04:50:01 on dist-test-slave-04bb"
server_key: "ff5f9f9e7cddde0311899de46796c9f0"
server_key_iv: "3705c1e93ecf5213fc8887c85f4eb3d1"
server_key_version: "encryptionkey@0"
I20260808 04:50:01.452749 22558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "ebcae3e621cc472ebf1bc8243a252cb9"
format_stamp: "Formatted at 2026-08-08 04:50:01 on dist-test-slave-04bb"
server_key: "ff5f9f9e7cddde0311899de46796c9f0"
server_key_iv: "3705c1e93ecf5213fc8887c85f4eb3d1"
server_key_version: "encryptionkey@0"
I20260808 04:50:01.456423 22558 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:50:01.458792 22576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:01.460000 22558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:50:01.460132 22558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "ebcae3e621cc472ebf1bc8243a252cb9"
format_stamp: "Formatted at 2026-08-08 04:50:01 on dist-test-slave-04bb"
server_key: "ff5f9f9e7cddde0311899de46796c9f0"
server_key_iv: "3705c1e93ecf5213fc8887c85f4eb3d1"
server_key_version: "encryptionkey@0"
I20260808 04:50:01.460247 22558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:01.478292 22558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:01.481684 22558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:01.481899 22558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:01.489231 22558 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:37945
I20260808 04:50:01.489256 22655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:37945 every 8 connection(s)
I20260808 04:50:01.490252 22558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:50:01.493507 22656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:01.495019 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22558
I20260808 04:50:01.495177 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:50:01.495484 22538 external_mini_cluster.cc:1428] Setting key d575b5b456f7f4293ba3b7ce4dbce3da
I20260808 04:50:01.500301 22656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9: Bootstrap starting.
I20260808 04:50:01.503013 22656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:01.503932 22656 log.cc:824] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:50:01 dist-test-slave-04bb krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:50:01.506282 22656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9: No bootstrap required, opened a new log
I20260808 04:50:01.511125 22656 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcae3e621cc472ebf1bc8243a252cb9" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37945 } }
I20260808 04:50:01.511402 22656 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:50:01.511452 22656 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebcae3e621cc472ebf1bc8243a252cb9, State: Initialized, Role: FOLLOWER
I20260808 04:50:01.511714 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:01.496974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56599 (local address 127.22.2.190:37945)
0808 04:50:01.497421 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:01.497442 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:01.497488 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:50:01.498364 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:01.498392 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:01.498838 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:01.499232 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:01.499980 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:01.501430 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:01.502385 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:01.503076 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:01.503243 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:01.506356 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:01.506363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:01.506380 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:01.506410 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:01.508416 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:01.508930 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:01.508937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:01.508943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:01.509090 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:01.509382 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:01.509386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:01.509388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:01.509910 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:01.510110 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:01.510390 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 04:50:01.510714 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":118,"threads_started":1}
I20260808 04:50:01.511951 22656 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcae3e621cc472ebf1bc8243a252cb9" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37945 } }
I20260808 04:50:01.512112 22656 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:50:01.512177 22656 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:50:01.512272 22656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:01.513285 22656 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcae3e621cc472ebf1bc8243a252cb9" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37945 } }
I20260808 04:50:01.513667 22656 leader_election.cc:302] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebcae3e621cc472ebf1bc8243a252cb9; no voters:
I20260808 04:50:01.514032 22656 leader_election.cc:288] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:50:01.514106 22662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:50:01.514355 22662 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [term 1 LEADER]: Becoming Leader. State: Replica: ebcae3e621cc472ebf1bc8243a252cb9, State: Running, Role: LEADER
I20260808 04:50:01.514671 22662 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcae3e621cc472ebf1bc8243a252cb9" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37945 } }
I20260808 04:50:01.514750 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:50:01.515436 22656 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:50:01.516356 22666 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcae3e621cc472ebf1bc8243a252cb9. Latest consensus state: current_term: 1 leader_uuid: "ebcae3e621cc472ebf1bc8243a252cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcae3e621cc472ebf1bc8243a252cb9" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37945 } } }
I20260808 04:50:01.516496 22666 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:01.516832 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebcae3e621cc472ebf1bc8243a252cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcae3e621cc472ebf1bc8243a252cb9" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37945 } } }
I20260808 04:50:01.516938 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:01.517220 22683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:50:01.521159 22683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:50:01.526525 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:01.517151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52703 (local address 127.22.2.190:37945)
0808 04:50:01.517276 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:01.517282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:01.517781 (+ 499us) server_negotiation.cc:407] Connection header received
0808 04:50:01.517977 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:01.517980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:01.518032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:01.518131 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:01.519065 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:01.519809 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:01.520982 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:01.521318 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:01.521431 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:01.523064 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:01.523069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:01.523071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:01.523098 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:01.524762 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:01.525257 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:01.525261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:01.525262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:01.525353 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:01.525693 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:01.525696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:01.525698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:01.525877 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:01.526013 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:01.526285 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 04:50:01.526407 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 04:50:01.527992 22683 catalog_manager.cc:1347] Generated new cluster ID: 160c7970effd480d9b1072e62d8ddf12
I20260808 04:50:01.528069 22683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:50:01.535910 22683 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:50:01.536793 22683 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:50:01.543277 22683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebcae3e621cc472ebf1bc8243a252cb9: Generated new TSK 0
I20260808 04:50:01.544055 22683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:50:01.553651 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22558
2026-08-08T04:50:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1633 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:50:01 dist-test-slave-04bb krb5kdc[22704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:50:01 dist-test-slave-04bb krb5kdc[22704](info): set up 2 sockets
Aug 08 04:50:01 dist-test-slave-04bb krb5kdc[22704](info): commencing operation
krb5kdc: starting...
W20260808 04:50:02.793371 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:50:02 dist-test-slave-04bb krb5kdc[22704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:50:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:50:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:50:02.921273 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41531
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:36185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:50:03.033859 22723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:03.034142 22723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:03.034212 22723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:50:03.038132 22723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:03.038213 22723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:03.038254 22723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:03.038290 22723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:03.038332 22723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:03.043356 22723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:03.044380 22723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:03.045395 22723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:03.053792 22733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:03.053820 22729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:03.053834 22730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:03.054271 22723 server_base.cc:1034] running on GCE node
I20260808 04:50:03.054819 22723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:03.055843 22723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:03.056999 22723 hybrid_clock.cc:648] HybridClock initialized: now 1786164603056980 us; error 30 us; skew 500 ppm
Aug 08 04:50:03 dist-test-slave-04bb krb5kdc[22704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164603, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:03.064672 22723 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:50:03.065569 22723 webserver.cc:460] Webserver started at http://127.22.2.190:35741/ using document root <none> and password file <none>
I20260808 04:50:03.066131 22723 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:03.066190 22723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:03.066424 22723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:03.068418 22723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "c85559b686d945eaa870bcbf4775cfc6"
format_stamp: "Formatted at 2026-08-08 04:50:03 on dist-test-slave-04bb"
server_key: "cc7f64ac09deecec090146d3d6fdf064"
server_key_iv: "f0ddbdce3344f20521d398ca40eee939"
server_key_version: "encryptionkey@0"
I20260808 04:50:03.068995 22723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "c85559b686d945eaa870bcbf4775cfc6"
format_stamp: "Formatted at 2026-08-08 04:50:03 on dist-test-slave-04bb"
server_key: "cc7f64ac09deecec090146d3d6fdf064"
server_key_iv: "f0ddbdce3344f20521d398ca40eee939"
server_key_version: "encryptionkey@0"
I20260808 04:50:03.072577 22723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:50:03.074965 22742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:03.076171 22723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:50:03.076400 22723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "c85559b686d945eaa870bcbf4775cfc6"
format_stamp: "Formatted at 2026-08-08 04:50:03 on dist-test-slave-04bb"
server_key: "cc7f64ac09deecec090146d3d6fdf064"
server_key_iv: "f0ddbdce3344f20521d398ca40eee939"
server_key_version: "encryptionkey@0"
I20260808 04:50:03.076514 22723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:03.096072 22723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:03.099177 22723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:03.099435 22723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:03.106770 22723 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41531
I20260808 04:50:03.106799 22837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41531 every 8 connection(s)
I20260808 04:50:03.107728 22723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:50:03.110597 22838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:03.116110 22838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6: Bootstrap starting.
I20260808 04:50:03.117358 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22723
I20260808 04:50:03.117509 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:50:03.117789 22538 external_mini_cluster.cc:1428] Setting key e6554e8623f4c6c6232b6cf9fcd7da4e
I20260808 04:50:03.119066 22838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:03.119840 22838 log.cc:824] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6: Log is configured to *not* fsync() on all Append() calls
I20260808 04:50:03.121768 22838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6: No bootstrap required, opened a new log
I20260808 04:50:03.125126 22838 raft_consensus.cc:348] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85559b686d945eaa870bcbf4775cfc6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41531 } }
I20260808 04:50:03.125363 22838 raft_consensus.cc:374] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:50:03.125428 22838 raft_consensus.cc:729] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c85559b686d945eaa870bcbf4775cfc6, State: Initialized, Role: FOLLOWER
I20260808 04:50:03.125939 22838 consensus_queue.cc:260] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85559b686d945eaa870bcbf4775cfc6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41531 } }
I20260808 04:50:03.126070 22838 raft_consensus.cc:388] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:50:03.126170 22838 raft_consensus.cc:482] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:50:03.126303 22838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:03.126765 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:03.118938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48509 (local address 127.22.2.190:41531)
0808 04:50:03.119418 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:03.119433 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:03.119469 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:50:03.120141 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:03.120161 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:03.120649 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:03.120984 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:03.121404 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:03.122406 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:03.123331 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:03.123789 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:03.123958 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:03.124814 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:03.124820 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:03.124832 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:50:03.125198 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:03.125249 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:50:03.125271 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:03.125401 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:03.125584 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 04:50:03.125809 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":126,"threads_started":1}
I20260808 04:50:03.127221 22838 raft_consensus.cc:504] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85559b686d945eaa870bcbf4775cfc6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41531 } }
I20260808 04:50:03.127565 22838 leader_election.cc:302] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c85559b686d945eaa870bcbf4775cfc6; no voters:
I20260808 04:50:03.127877 22838 leader_election.cc:288] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:50:03.127975 22845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:50:03.128233 22845 raft_consensus.cc:686] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [term 1 LEADER]: Becoming Leader. State: Replica: c85559b686d945eaa870bcbf4775cfc6, State: Running, Role: LEADER
I20260808 04:50:03.128897 22845 consensus_queue.cc:237] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85559b686d945eaa870bcbf4775cfc6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41531 } }
I20260808 04:50:03.129194 22838 sys_catalog.cc:564] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:50:03.129736 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:50:03.130136 22845 sys_catalog.cc:455] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c85559b686d945eaa870bcbf4775cfc6. Latest consensus state: current_term: 1 leader_uuid: "c85559b686d945eaa870bcbf4775cfc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85559b686d945eaa870bcbf4775cfc6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41531 } } }
I20260808 04:50:03.130318 22845 sys_catalog.cc:458] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:03.130741 22857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:50:03.131870 22846 sys_catalog.cc:455] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c85559b686d945eaa870bcbf4775cfc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85559b686d945eaa870bcbf4775cfc6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41531 } } }
I20260808 04:50:03.131995 22846 sys_catalog.cc:458] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6 [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:03.134555 22857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:50:03.136847 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:03.132434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50215 (local address 127.22.2.190:41531)
0808 04:50:03.132554 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:03.132560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:03.132743 (+ 183us) server_negotiation.cc:407] Connection header received
0808 04:50:03.132971 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:03.132974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:03.133029 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:03.133086 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:03.133559 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:03.134054 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:03.135198 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:03.135433 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:03.135545 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:03.136361 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:03.136365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:03.136367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:50:03.136379 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:03.136410 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:50:03.136420 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:03.136497 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:03.136654 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 04:50:03.136710 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 04:50:03.140885 22857 catalog_manager.cc:1347] Generated new cluster ID: 0bd2490fdde14ea98b8e632a0ceeda39
I20260808 04:50:03.140960 22857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:50:03.151983 22857 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:50:03.152902 22857 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:50:03.157713 22857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c85559b686d945eaa870bcbf4775cfc6: Generated new TSK 0
I20260808 04:50:03.158349 22857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:50:03.167423 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22723
2026-08-08T04:50:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:50:03 dist-test-slave-04bb krb5kdc[22882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:50:03 dist-test-slave-04bb krb5kdc[22882](info): set up 2 sockets
Aug 08 04:50:03 dist-test-slave-04bb krb5kdc[22882](info): commencing operation
krb5kdc: starting...
W20260808 04:50:04.450827 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:50:04 dist-test-slave-04bb krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:50:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:50:04.575456 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44793
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:44793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:50:04.704421 22899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:04.704743 22899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:04.704787 22899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:50:04.708714 22899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:04.708766 22899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:04.708825 22899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:04.708843 22899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:04.708859 22899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:04.713965 22899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:44793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:04.714898 22899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:04.715835 22899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:04.724072 22912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:04.724083 22909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:04.724102 22907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:04.724561 22899 server_base.cc:1034] running on GCE node
I20260808 04:50:04.725111 22899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:04.726227 22899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:04.727418 22899 hybrid_clock.cc:648] HybridClock initialized: now 1786164604727403 us; error 27 us; skew 500 ppm
Aug 08 04:50:04 dist-test-slave-04bb krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164604, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:04.730157 22899 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:50:04.731050 22899 webserver.cc:460] Webserver started at http://127.22.2.190:42837/ using document root <none> and password file <none>
I20260808 04:50:04.731606 22899 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:04.731695 22899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:04.731946 22899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:04.734061 22899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "007fadffa94b443da386ac6c9709363a"
format_stamp: "Formatted at 2026-08-08 04:50:04 on dist-test-slave-04bb"
server_key: "a5191c02961c28083b5970d8c83aeca1"
server_key_iv: "622ffb167b783220f41c6a7ed7d197b6"
server_key_version: "encryptionkey@0"
I20260808 04:50:04.734632 22899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "007fadffa94b443da386ac6c9709363a"
format_stamp: "Formatted at 2026-08-08 04:50:04 on dist-test-slave-04bb"
server_key: "a5191c02961c28083b5970d8c83aeca1"
server_key_iv: "622ffb167b783220f41c6a7ed7d197b6"
server_key_version: "encryptionkey@0"
I20260808 04:50:04.738371 22899 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:50:04.740793 22921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:04.742314 22899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 04:50:04.742452 22899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "007fadffa94b443da386ac6c9709363a"
format_stamp: "Formatted at 2026-08-08 04:50:04 on dist-test-slave-04bb"
server_key: "a5191c02961c28083b5970d8c83aeca1"
server_key_iv: "622ffb167b783220f41c6a7ed7d197b6"
server_key_version: "encryptionkey@0"
I20260808 04:50:04.742578 22899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:04.747411 22899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:04.750180 22899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:04.750322 22899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:04.757202 22899 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:44793
I20260808 04:50:04.757288 22999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:44793 every 8 connection(s)
I20260808 04:50:04.758154 22899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:50:04.761018 23001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:04.761523 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 22899
I20260808 04:50:04.761672 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:50:04.761979 22538 external_mini_cluster.cc:1428] Setting key 8f333628bc36022211735af2e210c68b
I20260808 04:50:04.767097 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:04.763322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52085 (local address 127.22.2.190:44793)
0808 04:50:04.763711 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:04.763727 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:04.763763 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:50:04.764425 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:04.764441 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:04.764770 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:04.765054 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:04.765899 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:04.765904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:04.765916 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:50:04.765927 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:04.765979 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:50:04.765997 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:04.766107 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:04.766224 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 04:50:04.766400 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":129,"threads_started":1}
I20260808 04:50:04.768316 23001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a: Bootstrap starting.
I20260808 04:50:04.770038 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:50:04.771059 23001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:04.772008 23001 log.cc:824] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a: Log is configured to *not* fsync() on all Append() calls
I20260808 04:50:04.774341 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:04.772401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44581 (local address 127.22.2.190:44793)
0808 04:50:04.772520 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:04.772543 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:04.772812 (+ 269us) server_negotiation.cc:407] Connection header received
0808 04:50:04.772953 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:04.772956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:04.772994 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:04.773085 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:04.773875 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:04.773879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:04.773881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:50:04.773893 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:04.773921 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:50:04.773934 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:04.774028 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:04.774182 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 04:50:04.774227 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 04:50:04.774462 23001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a: No bootstrap required, opened a new log
I20260808 04:50:04.778085 23001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007fadffa94b443da386ac6c9709363a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44793 } }
I20260808 04:50:04.778313 23001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:50:04.778350 23001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 007fadffa94b443da386ac6c9709363a, State: Initialized, Role: FOLLOWER
I20260808 04:50:04.778769 23001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007fadffa94b443da386ac6c9709363a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44793 } }
I20260808 04:50:04.778894 23001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:50:04.778941 23001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:50:04.779013 23001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:04.779902 23001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007fadffa94b443da386ac6c9709363a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44793 } }
I20260808 04:50:04.780254 23001 leader_election.cc:302] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 007fadffa94b443da386ac6c9709363a; no voters:
I20260808 04:50:04.780637 23001 leader_election.cc:288] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:50:04.780730 23015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:50:04.780987 23015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [term 1 LEADER]: Becoming Leader. State: Replica: 007fadffa94b443da386ac6c9709363a, State: Running, Role: LEADER
I20260808 04:50:04.781360 23015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007fadffa94b443da386ac6c9709363a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44793 } }
I20260808 04:50:04.781917 23001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:50:04.783062 23016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "007fadffa94b443da386ac6c9709363a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007fadffa94b443da386ac6c9709363a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44793 } } }
I20260808 04:50:04.783200 23016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:04.783597 23026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:50:04.783556 23017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 007fadffa94b443da386ac6c9709363a. Latest consensus state: current_term: 1 leader_uuid: "007fadffa94b443da386ac6c9709363a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007fadffa94b443da386ac6c9709363a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44793 } } }
I20260808 04:50:04.784346 23017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:04.787254 23026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:50:04.793390 23026 catalog_manager.cc:1347] Generated new cluster ID: 42c100c2fdc641da8c034525a34f49bc
I20260808 04:50:04.793463 23026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:50:04.810544 23026 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:50:04.810907 23026 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:50:04.820176 23026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 007fadffa94b443da386ac6c9709363a: Generated new TSK 0
I20260808 04:50:04.820986 23026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:50:04.829957 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 22899
2026-08-08T04:50:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1668 ms)
[----------] 18 tests from AuthTokenIssuingTest (29375 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:50:04 dist-test-slave-04bb krb5kdc[23043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:50:04 dist-test-slave-04bb krb5kdc[23043](info): set up 2 sockets
Aug 08 04:50:04 dist-test-slave-04bb krb5kdc[23043](info): commencing operation
krb5kdc: starting...
W20260808 04:50:06.143752 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:50:06.271291 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38865
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38865,127.22.2.189:34787,127.22.2.188:38737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:50:06.383049 23060 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 04:50:06.383379 23060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:06.383437 23060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:06.383509 23060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:50:06.387530 23060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:06.387612 23060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:06.387651 23060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:06.387691 23060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:06.387722 23060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:06.392903 23060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38865,127.22.2.189:34787,127.22.2.188:38737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:06.393940 23060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:06.394941 23060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:06.403470 23068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:06.403481 23070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:06.403456 23067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:06.403626 23060 server_base.cc:1034] running on GCE node
I20260808 04:50:06.404275 23060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:06.405392 23060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:06.406579 23060 hybrid_clock.cc:648] HybridClock initialized: now 1786164606406566 us; error 30 us; skew 500 ppm
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:06.409507 23060 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:50:06.410328 23060 webserver.cc:460] Webserver started at http://127.22.2.190:33709/ using document root <none> and password file <none>
I20260808 04:50:06.410838 23060 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:06.410897 23060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:06.411058 23060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:06.413058 23060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "eb86c1d824af40bcb2726cc0351a3f06"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "67365fd72dea181cf545bff814028b13"
server_key_iv: "af67a9d36345fe79660eb83f990a3937"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.413573 23060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "eb86c1d824af40bcb2726cc0351a3f06"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "67365fd72dea181cf545bff814028b13"
server_key_iv: "af67a9d36345fe79660eb83f990a3937"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.417287 23060 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 04:50:06.419674 23080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:06.420989 23060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:50:06.421113 23060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "eb86c1d824af40bcb2726cc0351a3f06"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "67365fd72dea181cf545bff814028b13"
server_key_iv: "af67a9d36345fe79660eb83f990a3937"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.421216 23060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:06.465328 23060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:06.468750 23060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:06.468963 23060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:06.476073 23060 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:38865
I20260808 04:50:06.476393 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:38865 every 8 connection(s)
I20260808 04:50:06.477092 23060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:50:06.478142 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23060
I20260808 04:50:06.478242 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:50:06.478542 22538 external_mini_cluster.cc:1428] Setting key 4d1c75fd07c03236df6f95d23e28a139
I20260808 04:50:06.480123 23170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:06.483987 23170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.2.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.189@KRBTEST.COM" created.
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:50:06.499873 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.486362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38865 (local address 127.0.0.1:38414)
0808 04:50:06.486949 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.486988 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.487864 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.488665 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.488670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.488996 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.489247 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.489260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.490687 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.490694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.491442 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.491452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.492420 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.492426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.492606 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.493232 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.493269 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.494907 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.496881 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.496890 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.496900 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.497127 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.497450 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.497453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.497455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.497566 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.498155 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.498162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.498305 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.498468 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.498859 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":31,"spinlock_wait_cycles":14208,"thread_start_us":177,"threads_started":1}
I20260808 04:50:06.499873 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.486361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.22.2.190:38865)
0808 04:50:06.486777 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.486798 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.487095 (+ 297us) server_negotiation.cc:407] Connection header received
0808 04:50:06.488176 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.488180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.488356 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.488674 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.489413 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.490559 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.491570 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.492223 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.492427 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.495054 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.495060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.495069 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.495089 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.496759 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.497237 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.497243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.497245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.497350 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.497676 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.497680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.497682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.498032 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.498181 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.498543 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.498913 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":275,"thread_start_us":139,"threads_started":1}
I20260808 04:50:06.502442 23170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:06.503103 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.502848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:53978)
0808 04:50:06.502970 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.503044 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 04:50:06.503295 23081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
I20260808 04:50:06.504338 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.504070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:53980)
0808 04:50:06.504166 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.504245 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 04:50:06.504601 23170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:34787: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
I20260808 04:50:06.505787 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:34787
--webserver_interface=127.22.2.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38865,127.22.2.189:34787,127.22.2.188:38737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:50:06.553845 23170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } attempt: 1
I20260808 04:50:06.554783 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.554473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:53990)
0808 04:50:06.554637 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.554705 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 04:50:06.555533 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.555320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:54006)
0808 04:50:06.555430 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.555481 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 04:50:06.555655 23170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:34787: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
W20260808 04:50:06.621503 23180 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 04:50:06.621830 23180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:06.621898 23180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:06.621976 23180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 04:50:06.623831 23170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } attempt: 2
I20260808 04:50:06.624804 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.624422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:54010)
0808 04:50:06.624623 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.624720 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 04:50:06.625639 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.625403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:54014)
0808 04:50:06.625529 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.625586 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 04:50:06.625803 23170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:34787: Network error: Client connection negotiation failed: client connection to 127.22.2.189:34787: connect: Connection refused (error 111)
W20260808 04:50:06.626396 23180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:06.626449 23180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:06.626468 23180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:06.626528 23180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:06.626573 23180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:06.631940 23180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38865,127.22.2.189:34787,127.22.2.188:38737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:34787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.2.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:06.633080 23180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:06.634119 23180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:06.642647 23186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:06.642696 23188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:06.642661 23185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:06.643003 23180 server_base.cc:1034] running on GCE node
I20260808 04:50:06.643560 23180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:06.644675 23180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:06.645855 23180 hybrid_clock.cc:648] HybridClock initialized: now 1786164606645840 us; error 33 us; skew 500 ppm
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:06.648669 23180 init.cc:377] Logged in from keytab as kudu/127.22.2.189@KRBTEST.COM (short username kudu)
I20260808 04:50:06.649504 23180 webserver.cc:460] Webserver started at http://127.22.2.189:45255/ using document root <none> and password file <none>
I20260808 04:50:06.650061 23180 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:06.650164 23180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:06.650396 23180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:06.652374 23180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/data/instance:
uuid: "aa68c3a8aa7c42a095d5525c4bc4339c"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "338fefa5dcc792a275114e256ea715a9"
server_key_iv: "4f7432e5776e268ddf40a4d98320b5a3"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.652988 23180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/wal/instance:
uuid: "aa68c3a8aa7c42a095d5525c4bc4339c"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "338fefa5dcc792a275114e256ea715a9"
server_key_iv: "4f7432e5776e268ddf40a4d98320b5a3"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.656512 23180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:50:06.658881 23199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:06.660077 23180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 04:50:06.660209 23180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/wal
uuid: "aa68c3a8aa7c42a095d5525c4bc4339c"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "338fefa5dcc792a275114e256ea715a9"
server_key_iv: "4f7432e5776e268ddf40a4d98320b5a3"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.660316 23180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:06.683316 23180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:06.686563 23180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:06.686769 23180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:06.693689 23180 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:34787
I20260808 04:50:06.693987 23271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:34787 every 8 connection(s)
I20260808 04:50:06.694649 23180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
I20260808 04:50:06.697479 23272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:06.701220 23272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:06.702524 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23180
I20260808 04:50:06.702670 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-1/wal/instance
I20260808 04:50:06.702965 22538 external_mini_cluster.cc:1428] Setting key 19a5c58ff6edb8885f3b640f448d3f83
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.188@KRBTEST.COM" created.
I20260808 04:50:06.715119 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.703201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.22.2.190:38865)
0808 04:50:06.703363 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.703368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.704242 (+ 874us) server_negotiation.cc:407] Connection header received
0808 04:50:06.705236 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.705240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.705320 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.705411 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.706552 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.707085 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.708219 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.708469 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.708607 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.711353 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.711356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.711358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.711383 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.712903 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.713461 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.713465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.713467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.713549 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.713883 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.713888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.713890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.714131 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.714240 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.714813 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.714940 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 04:50:06.715873 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.703578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38865 (local address 127.0.0.1:38418)
0808 04:50:06.704047 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.704101 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.705024 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.705583 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.705590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.705929 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.706405 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.706418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.707223 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.707226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.708086 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.708096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.708644 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.708648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.708820 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.709479 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.709496 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.711166 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.713079 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.713085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.713101 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.713356 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.713642 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.713646 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.713648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.713781 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.714254 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.714262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.714533 (+ 271us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.714748 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.715074 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":150,"threads_started":1}
I20260808 04:50:06.716962 23170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } attempt: 3
I20260808 04:50:06.717563 23272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 04:50:06.727398 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.718081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:54028)
0808 04:50:06.718734 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.718745 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.718832 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.719043 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.719045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.719197 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.719278 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.719282 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.720397 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.720400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.720955 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.720963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.721528 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.721532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.721617 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.722072 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.722083 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.723449 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.725740 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.725745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.725747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.725967 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.726284 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.726288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.726290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.726379 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.726841 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.726844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.726934 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.727046 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.727208 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606}
I20260808 04:50:06.727504 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.718234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.22.2.189:34787)
0808 04:50:06.718558 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.718568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.718809 (+ 241us) server_negotiation.cc:407] Connection header received
0808 04:50:06.718902 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.718907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.718945 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.719032 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.719388 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.720185 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.721064 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.721421 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.721533 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.723653 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.723666 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.723682 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.723712 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.725622 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.726089 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.726096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.726102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.726191 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.726464 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.726470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.726474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.726724 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.726873 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.727147 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.727290 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":126,"threads_started":1}
I20260808 04:50:06.729630 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.717396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:54016)
0808 04:50:06.717521 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.717540 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.717603 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.718604 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.718610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.718810 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.718967 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.718978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.720485 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.720491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.722076 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.722092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.722760 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.722766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.722899 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.723415 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.723431 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.724844 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.727949 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.727952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.727954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.728200 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.728578 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.728581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.728583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.728684 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.729082 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.729085 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.729191 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.729295 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.729457 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 04:50:06.729662 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.717850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.22.2.189:34787)
0808 04:50:06.718204 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.718213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.718231 (+ 18us) server_negotiation.cc:407] Connection header received
0808 04:50:06.718265 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.718270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.718451 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.718595 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.719124 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.720334 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.722238 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.722627 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.722751 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.724990 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.724993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.724996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.725020 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.727398 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.728319 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.728322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.728324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.728444 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.728803 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.728806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.728808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.728985 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.729064 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.729401 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.729502 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":200,"server-negotiator.queue_time_us":263,"thread_start_us":177,"threads_started":1}
I20260808 04:50:06.729699 23272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:06.730299 23170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:06.730458 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.730201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49552)
0808 04:50:06.730306 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.730392 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 04:50:06.730630 23200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
I20260808 04:50:06.731038 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.730784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49566)
0808 04:50:06.730909 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.730967 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 04:50:06.731797 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.731497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49582)
0808 04:50:06.731686 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.731738 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 04:50:06.731797 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.731500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49584)
0808 04:50:06.731684 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.731738 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 04:50:06.731933 23170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:38737: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
W20260808 04:50:06.732061 23272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:38737: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
I20260808 04:50:06.733578 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:38737
--webserver_interface=127.22.2.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38865,127.22.2.189:34787,127.22.2.188:38737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:50:06.763098 23170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } attempt: 1
I20260808 04:50:06.763969 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.763663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49586)
0808 04:50:06.763828 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.763889 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 04:50:06.764807 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.764561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49600)
0808 04:50:06.764654 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.764723 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 04:50:06.764938 23170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:38737: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
I20260808 04:50:06.781283 23272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } attempt: 1
I20260808 04:50:06.782105 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.781832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49614)
0808 04:50:06.781974 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.782034 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 04:50:06.782831 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.782612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49628)
0808 04:50:06.782718 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.782766 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 04:50:06.782961 23272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:38737: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
I20260808 04:50:06.840095 23170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } attempt: 2
I20260808 04:50:06.841041 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.840737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49642)
0808 04:50:06.840893 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.840959 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 04:50:06.841794 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.841571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49648)
0808 04:50:06.841675 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.841738 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 04:50:06.841928 23170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:38737: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
I20260808 04:50:06.851110 23272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } attempt: 2
I20260808 04:50:06.851917 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.851643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49654)
0808 04:50:06.851786 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.851848 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 04:50:06.852643 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.852411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49670)
0808 04:50:06.852517 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.852590 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 04:50:06.852761 23272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:38737: Network error: Client connection negotiation failed: client connection to 127.22.2.188:38737: connect: Connection refused (error 111)
W20260808 04:50:06.854960 23287 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 04:50:06.855258 23287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:06.855316 23287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:06.855387 23287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:50:06.859521 23287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:06.859592 23287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:06.859639 23287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:06.859676 23287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:06.859712 23287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:06.865140 23287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38865,127.22.2.189:34787,127.22.2.188:38737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:38737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.2.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:06.866166 23287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:06.867158 23287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:06.875753 23294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:06.875744 23295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:06.875794 23287 server_base.cc:1034] running on GCE node
W20260808 04:50:06.875744 23297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:06.876518 23287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:06.877627 23287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:06.878803 23287 hybrid_clock.cc:648] HybridClock initialized: now 1786164606878787 us; error 27 us; skew 500 ppm
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:06.881697 23287 init.cc:377] Logged in from keytab as kudu/127.22.2.188@KRBTEST.COM (short username kudu)
I20260808 04:50:06.882611 23287 webserver.cc:460] Webserver started at http://127.22.2.188:39735/ using document root <none> and password file <none>
I20260808 04:50:06.883190 23287 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:06.883255 23287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:06.883514 23287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:06.885707 23287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/data/instance:
uuid: "1a9d9eed33ad4483a4f07f03b1de49df"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "e1234ecd5dea0537254b97f06bec9e88"
server_key_iv: "36bd4fda5c78d124bdcc2f1ef1d4acfe"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.886296 23287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/wal/instance:
uuid: "1a9d9eed33ad4483a4f07f03b1de49df"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "e1234ecd5dea0537254b97f06bec9e88"
server_key_iv: "36bd4fda5c78d124bdcc2f1ef1d4acfe"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.890223 23287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 04:50:06.892735 23306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:06.893998 23287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 04:50:06.894153 23287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/wal
uuid: "1a9d9eed33ad4483a4f07f03b1de49df"
format_stamp: "Formatted at 2026-08-08 04:50:06 on dist-test-slave-04bb"
server_key: "e1234ecd5dea0537254b97f06bec9e88"
server_key_iv: "36bd4fda5c78d124bdcc2f1ef1d4acfe"
server_key_version: "encryptionkey@0"
I20260808 04:50:06.894304 23287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:06.917730 23287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:06.920832 23287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:06.921036 23287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:06.929409 23287 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:38737
I20260808 04:50:06.929688 23374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:38737 every 8 connection(s)
I20260808 04:50:06.930483 23287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
I20260808 04:50:06.933279 23375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:06.937170 23375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:06.940809 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23287
I20260808 04:50:06.940965 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786164310373382-22538-0/minicluster-data/master-2/wal/instance
I20260808 04:50:06.941084 23170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } attempt: 3
I20260808 04:50:06.941293 22538 external_mini_cluster.cc:1428] Setting key cb0964e777c02f1d0f61bdda41c6b4a2
I20260808 04:50:06.943935 23272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } attempt: 3
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:50:06.954766 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.939012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.22.2.190:38865)
0808 04:50:06.939156 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.939160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.939965 (+ 805us) server_negotiation.cc:407] Connection header received
0808 04:50:06.940910 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.940914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.940977 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.941077 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.943082 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.943585 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.945273 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.945619 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.945707 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.950230 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.950236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.950239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.950271 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.951905 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.952776 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.952780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.952782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.952875 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.953331 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.953336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.953341 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.953665 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.953759 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.954457 (+ 698us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.954594 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":1920}
I20260808 04:50:06.955142 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.944371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49700)
0808 04:50:06.944643 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.944657 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.944731 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.945399 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.945402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.945597 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.945702 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.945708 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.946830 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.946836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.947523 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.947533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.948169 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.948173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.948260 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.948798 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.948823 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.950615 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.953542 (+ 2927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.953545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.953547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.953798 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.954122 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.954125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.954127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.954228 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.954653 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.954656 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.954752 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.954852 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.954992 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260808 04:50:06.955224 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.941500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49686)
0808 04:50:06.941681 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.941698 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.941781 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.942503 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.942507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.942683 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.942778 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.942784 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.944600 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.944603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.945412 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.945423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.946308 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.946312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.946408 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.947217 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.947234 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.948892 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.952885 (+ 3993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.952889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.952891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.953170 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.953542 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.953545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.953547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.953654 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.954275 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.954278 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.954400 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.954488 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.955043 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":414,"spinlock_wait_cycles":896}
I20260808 04:50:06.956897 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.939375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38865 (local address 127.0.0.1:38430)
0808 04:50:06.939816 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.939859 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.940683 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.941263 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.941275 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.942231 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.942934 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.942956 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.943699 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.943703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.945133 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.945144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.945920 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.945923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.946208 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.947754 (+ 1546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.947773 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.950061 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.952036 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.952044 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.952055 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.952649 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.952998 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.953003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.953005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.953196 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.953823 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.953829 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.954165 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.955441 (+ 1276us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.955940 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":531,"spinlock_wait_cycles":3456,"thread_start_us":111,"threads_started":1}
I20260808 04:50:06.956897 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.942386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.22.2.188:38737)
0808 04:50:06.942633 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.942641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.942655 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:50:06.942687 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.942689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.942726 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.942864 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.943771 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.944615 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.945788 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.946277 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.946393 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.949674 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.949678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.949680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.949715 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.953318 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.953787 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.953791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.953808 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.953892 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.954197 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.954200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.954202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.954405 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.954591 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.955349 (+ 758us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.955871 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":520,"server-negotiator.queue_time_us":191,"thread_start_us":89,"threads_started":1}
I20260808 04:50:06.956920 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.944877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.22.2.188:38737)
0808 04:50:06.945153 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.945166 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.945179 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:50:06.945211 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.945214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.945250 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.945809 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.945927 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.946711 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.947646 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.948020 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.948501 (+ 481us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.951054 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.951059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.951062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.951092 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.953414 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.953933 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.953936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.953937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.954008 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.954344 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.954347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.954348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.954539 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.955456 (+ 917us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.955530 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.956030 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":420,"server-negotiator.queue_time_us":218,"thread_start_us":93,"threads_started":1}
I20260808 04:50:06.956920 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.941626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.22.2.188:38737)
0808 04:50:06.942008 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.942016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.942036 (+ 20us) server_negotiation.cc:407] Connection header received
0808 04:50:06.942072 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.942075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.942285 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.942922 (+ 637us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.943058 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.944434 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.945559 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.946166 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.946616 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.949064 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.949074 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.949089 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.949120 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.952750 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.953336 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.953343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.953346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.953432 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.953783 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.953787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.953788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.954148 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.954618 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.954735 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.955769 (+ 1034us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":291,"thread_start_us":195,"threads_started":1}
I20260808 04:50:06.958971 23375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:06.961930 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06: Bootstrap starting.
I20260808 04:50:06.962003 23272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c: Bootstrap starting.
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 04:50:06.964926 23170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:06.964977 23272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:06.965875 23170 log.cc:824] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06: Log is configured to *not* fsync() on all Append() calls
I20260808 04:50:06.965914 23272 log.cc:824] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c: Log is configured to *not* fsync() on all Append() calls
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:50:06.968070 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06: No bootstrap required, opened a new log
I20260808 04:50:06.968070 23272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c: No bootstrap required, opened a new log
I20260808 04:50:06.969280 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.959422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:34787 (local address 127.0.0.1:54042)
0808 04:50:06.959864 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.959877 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.959979 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.960285 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.960288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.960469 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.960611 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.960619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.961628 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.961631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.962422 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.962433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.963128 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.963133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.963248 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.963703 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.963717 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.965255 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.967356 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.967359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.967361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.967608 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.967982 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.967986 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.967989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.968091 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.968711 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.968715 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.968842 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.968962 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.969094 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379}
I20260808 04:50:06.969777 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.959668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.22.2.189:34787)
0808 04:50:06.959778 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.959783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.959926 (+ 143us) server_negotiation.cc:407] Connection header received
0808 04:50:06.960071 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.960074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.960117 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.960164 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.960750 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.961481 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.962563 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.962898 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.962975 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.965424 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.965427 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.965430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.965454 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.967219 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.967729 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.967735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.967740 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.967853 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.968212 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.968216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.968219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.968550 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.968658 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.969485 (+ 827us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.969625 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 04:50:06.970470 23375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:06.971946 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.959596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.22.2.190:38865)
0808 04:50:06.959705 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.959709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.959925 (+ 216us) server_negotiation.cc:407] Connection header received
0808 04:50:06.960086 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.960090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.960131 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.960178 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.961055 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.961766 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.962950 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.963289 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.963383 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.967498 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.967523 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.967526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.967554 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.969462 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.970487 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.970491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.970493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.970586 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.970888 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.970892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.970894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.971116 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.971277 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.971649 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.971764 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 04:50:06.972294 23170 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } }
I20260808 04:50:06.972586 23170 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb86c1d824af40bcb2726cc0351a3f06, State: Initialized, Role: FOLLOWER
I20260808 04:50:06.972941 23170 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } }
I20260808 04:50:06.972963 23272 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } }
I20260808 04:50:06.973153 23272 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa68c3a8aa7c42a095d5525c4bc4339c, State: Initialized, Role: FOLLOWER
I20260808 04:50:06.973490 23272 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } }
I20260808 04:50:06.973469 23384 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:06.973728 23384 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:06.974376 23170 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:50:06.974726 23272 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:50:06.977483 23385 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:06.977608 23385 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
Aug 08 04:50:06 dist-test-slave-04bb krb5kdc[23043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164606, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:50:06.984198 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.973437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54054 (local address 127.22.2.189:34787)
0808 04:50:06.973542 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.973545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.973715 (+ 170us) server_negotiation.cc:407] Connection header received
0808 04:50:06.973907 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.973911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.973952 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.973998 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.974689 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.975287 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.976079 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.976428 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.976562 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.979410 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.979414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.979417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.979442 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.981297 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.982098 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.982102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.982104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.982188 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.982521 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.982525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.982527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.982772 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.982961 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.983573 (+ 612us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.984004 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 04:50:06.985009 23422 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:50:06.985093 23422 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 04:50:06.985522 23421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:50:06.985605 23421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:50:06.983116 23180 master_runner.cc:186] Error getting master registration for 127.22.2.188:38737: OK, instance_id { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" instance_seqno: 1786164606927496 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 04:50:06.986390 23180 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 04:50:06.986951 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.970946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:38737 (local address 127.0.0.1:49708)
0808 04:50:06.971239 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:06.971252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:06.971323 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:06.976924 (+ 5601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:06.976929 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:06.977117 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:06.977222 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.977232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.979295 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.979301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.980213 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:06.980225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.981007 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.981014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:06.981109 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.981774 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:06.981793 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:06.983321 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:06.985396 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.985401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.985403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.985612 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.985891 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:06.985894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:06.985896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:06.985981 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:06.986403 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:06.986406 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:06.986518 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 04:50:06.986630 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 04:50:06.986776 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234}
I20260808 04:50:06.986954 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:06.971139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.22.2.188:38737)
0808 04:50:06.976635 (+ 5496us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:06.976641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:06.976660 (+ 19us) server_negotiation.cc:407] Connection header received
0808 04:50:06.976700 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:06.976704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:06.976753 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:06.976898 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:06.978223 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.979022 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.980364 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:06.980791 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:06.980901 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:06.983457 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:06.983460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:06.983463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:06.983487 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:06.985292 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.985725 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.985728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.985730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.985799 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:06.986064 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:06.986069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:06.986070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:06.986237 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:06.986333 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:06.986696 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 04:50:06.986831 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":5407}
I20260808 04:50:06.990033 23375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df: Bootstrap starting.
I20260808 04:50:06.992235 23375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:06.988868 23060 master_runner.cc:186] Error getting master registration for 127.22.2.188:38737: OK, instance_id { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" instance_seqno: 1786164606927496 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 04:50:06.992966 23060 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 04:50:06.993156 23375 log.cc:824] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df: Log is configured to *not* fsync() on all Append() calls
I20260808 04:50:06.994861 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:50:06.996281 23130 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "eb86c1d824af40bcb2726cc0351a3f06"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38434
I20260808 04:50:06.996624 23130 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 04:50:06.996767 23130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:06.998160 23130 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } }
I20260808 04:50:06.998631 23375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df: No bootstrap required, opened a new log
I20260808 04:50:06.999171 23130 leader_election.cc:288] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [CANDIDATE]: Term 1 election: Requested vote from peers aa68c3a8aa7c42a095d5525c4bc4339c (127.22.2.189:34787), 1a9d9eed33ad4483a4f07f03b1de49df (127.22.2.188:38737)
I20260808 04:50:07.000401 23237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb86c1d824af40bcb2726cc0351a3f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c"
I20260808 04:50:07.000680 23237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:07.002081 23237 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb86c1d824af40bcb2726cc0351a3f06 in term 1.
I20260808 04:50:07.002928 23081 leader_election.cc:302] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa68c3a8aa7c42a095d5525c4bc4339c, eb86c1d824af40bcb2726cc0351a3f06; no voters:
I20260808 04:50:07.003225 23384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:50:07.003464 23384 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [term 1 LEADER]: Becoming Leader. State: Replica: eb86c1d824af40bcb2726cc0351a3f06, State: Running, Role: LEADER
I20260808 04:50:07.003816 23384 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } }
I20260808 04:50:07.004305 23375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } }
I20260808 04:50:07.004626 23375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9d9eed33ad4483a4f07f03b1de49df, State: Initialized, Role: FOLLOWER
I20260808 04:50:07.005023 23375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } }
I20260808 04:50:07.005677 23384 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb86c1d824af40bcb2726cc0351a3f06. Latest consensus state: current_term: 1 leader_uuid: "eb86c1d824af40bcb2726cc0351a3f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:07.005821 23384 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:07.006583 23375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:50:07.007556 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb86c1d824af40bcb2726cc0351a3f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1a9d9eed33ad4483a4f07f03b1de49df"
I20260808 04:50:07.007803 23345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:07.009099 23345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb86c1d824af40bcb2726cc0351a3f06 in term 1.
I20260808 04:50:07.010500 23435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:50:07.013743 23435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:50:07.016700 23430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:07.016837 23430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [sys.catalog]: This master's current role is: FOLLOWER
W20260808 04:50:07.018981 23457 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:50:07.019061 23457 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:50:07.030076 23345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [term 1 FOLLOWER]: Refusing update from remote peer eb86c1d824af40bcb2726cc0351a3f06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:50:07.030119 23237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [term 1 FOLLOWER]: Refusing update from remote peer eb86c1d824af40bcb2726cc0351a3f06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:50:07.030823 23428 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:50:07.031026 23384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:50:07.034798 23385 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb86c1d824af40bcb2726cc0351a3f06. Latest consensus state: current_term: 1 leader_uuid: "eb86c1d824af40bcb2726cc0351a3f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:07.034940 23385 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:07.035079 23430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb86c1d824af40bcb2726cc0351a3f06. Latest consensus state: current_term: 1 leader_uuid: "eb86c1d824af40bcb2726cc0351a3f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:07.035199 23430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:07.042009 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:07.011453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.22.2.190:38865)
0808 04:50:07.011631 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.011635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.020858 (+ 9223us) server_negotiation.cc:407] Connection header received
0808 04:50:07.020989 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.020993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.021047 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.021129 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.028065 (+ 6936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:07.029223 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:07.036065 (+ 6842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:07.036411 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:07.036516 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:07.038290 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:07.038294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:07.038297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:07.038326 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:07.040075 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:07.040589 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:07.040593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:07.040595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:07.040682 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:07.041030 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:07.041033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:07.041035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:07.041246 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:07.041361 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:07.041710 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 04:50:07.041848 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 04:50:07.049643 23384 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb86c1d824af40bcb2726cc0351a3f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:07.049793 23384 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:07.049928 23430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb86c1d824af40bcb2726cc0351a3f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:07.050037 23430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:07.050207 23458 mvcc.cc:205] Tried to move back new op lower bound from 7316130230380941312 to 7316130230291415040. Current Snapshot: MvccSnapshot[applied={T|T < 7316130230380941312}]
I20260808 04:50:07.051453 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:07.016130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54068 (local address 127.22.2.189:34787)
0808 04:50:07.016324 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.016329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.021173 (+ 4844us) server_negotiation.cc:407] Connection header received
0808 04:50:07.021569 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.021572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.021622 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.021671 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.024192 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:07.025045 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:07.027507 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:07.027881 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:07.027963 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:07.038272 (+ 10309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:07.038276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:07.038279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:07.038308 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:07.040775 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:07.042220 (+ 1445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:07.042224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:07.042226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:07.042316 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:07.042620 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:07.042624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:07.042625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:07.042854 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:07.042975 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:07.051132 (+ 8157us) server_negotiation.cc:299] Negotiation successful
0808 04:50:07.051277 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260808 04:50:07.054171 23428 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb86c1d824af40bcb2726cc0351a3f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:07.054287 23428 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06 [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:07.055184 23385 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb86c1d824af40bcb2726cc0351a3f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb86c1d824af40bcb2726cc0351a3f06" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38865 } } peers { permanent_uuid: "aa68c3a8aa7c42a095d5525c4bc4339c" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 34787 } } peers { permanent_uuid: "1a9d9eed33ad4483a4f07f03b1de49df" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 38737 } } }
I20260808 04:50:07.055295 23385 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:07.055681 23466 mvcc.cc:205] Tried to move back new op lower bound from 7316130230380941312 to 7316130230291415040. Current Snapshot: MvccSnapshot[applied={T|T < 7316130230380941312}]
I20260808 04:50:07.058596 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:07.016449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.22.2.188:38737)
0808 04:50:07.016565 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.016569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.021417 (+ 4848us) server_negotiation.cc:407] Connection header received
0808 04:50:07.021519 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.021522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.021564 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.021636 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.026165 (+ 4529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:07.026920 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:07.037198 (+ 10278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:07.037543 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:07.037615 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:07.050740 (+ 13125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:07.050744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:07.050747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:07.050775 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:07.056593 (+ 5818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:07.057119 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:07.057123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:07.057125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:07.057222 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:07.057555 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:07.057558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:07.057560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:07.057798 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:07.057917 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:07.058296 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 04:50:07.058431 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 04:50:07.061470 23435 catalog_manager.cc:1347] Generated new cluster ID: 0db7896f5ccd40a7947b584647915193
I20260808 04:50:07.061547 23435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:50:07.061928 23468 mvcc.cc:205] Tried to move back new op lower bound from 7316130230380941312 to 7316130230291415040. Current Snapshot: MvccSnapshot[applied={T|T < 7316130230380941312}]
I20260808 04:50:07.093918 23435 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:50:07.095259 23435 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:50:07.118806 23435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb86c1d824af40bcb2726cc0351a3f06: Generated new TSK 0
I20260808 04:50:07.122037 23435 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 04:50:07.164386 23175 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.160501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.22.2.190:38865)
0808 04:50:07.160732 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.160737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.162972 (+ 2235us) server_negotiation.cc:407] Connection header received
0808 04:50:07.163285 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.163290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.163348 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.163412 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.164231 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38458: BlockingRecv error: recv got EOF from 127.0.0.1:38458 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
W20260808 04:50:07.166548 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.161399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54078 (local address 127.22.2.189:34787)
0808 04:50:07.161522 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.161527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.165264 (+ 3737us) server_negotiation.cc:407] Connection header received
0808 04:50:07.165735 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.165739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.165794 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.165857 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.166418 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54078: BlockingRecv error: recv got EOF from 127.0.0.1:54078 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 04:50:07.168059 23380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.162290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.22.2.188:38737)
0808 04:50:07.162428 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.162433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.165632 (+ 3199us) server_negotiation.cc:407] Connection header received
0808 04:50:07.167124 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.167128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.167188 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.167252 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.167930 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49726: BlockingRecv error: recv got EOF from 127.0.0.1:49726 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 04:50:07.174427 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.171765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.22.2.189:34787)
0808 04:50:07.171877 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.171882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.173369 (+ 1487us) server_negotiation.cc:407] Connection header received
0808 04:50:07.173611 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.173615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.173706 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.173770 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.174322 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54092: BlockingRecv error: recv got EOF from 127.0.0.1:54092 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 04:50:07.179908 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.178438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.22.2.189:34787)
0808 04:50:07.178536 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.178540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.179028 (+ 488us) server_negotiation.cc:407] Connection header received
0808 04:50:07.179206 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.179209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.179258 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.179310 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.179814 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54098: BlockingRecv error: recv got EOF from 127.0.0.1:54098 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 04:50:07.185972 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.184517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54108 (local address 127.22.2.189:34787)
0808 04:50:07.184645 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.184648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.185139 (+ 491us) server_negotiation.cc:407] Connection header received
0808 04:50:07.185315 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.185318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.185364 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.185411 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.185882 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54108: BlockingRecv error: recv got EOF from 127.0.0.1:54108 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 04:50:07.193801 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.192368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.22.2.189:34787)
0808 04:50:07.192482 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.192486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.192956 (+ 470us) server_negotiation.cc:407] Connection header received
0808 04:50:07.193148 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.193152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.193198 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.193245 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.193708 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54120: BlockingRecv error: recv got EOF from 127.0.0.1:54120 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 04:50:07.205354 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.204220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.22.2.189:34787)
0808 04:50:07.204365 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.204370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.204505 (+ 135us) server_negotiation.cc:407] Connection header received
0808 04:50:07.204689 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.204693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.204749 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.204832 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.205244 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54122: BlockingRecv error: recv got EOF from 127.0.0.1:54122 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 04:50:07.224421 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.223295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54128 (local address 127.22.2.189:34787)
0808 04:50:07.223433 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.223437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.223571 (+ 134us) server_negotiation.cc:407] Connection header received
0808 04:50:07.223690 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.223694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.223758 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.223863 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.224305 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54128: BlockingRecv error: recv got EOF from 127.0.0.1:54128 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 04:50:07.259490 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.258369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.22.2.189:34787)
0808 04:50:07.258565 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.258569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.258619 (+ 50us) server_negotiation.cc:407] Connection header received
0808 04:50:07.258763 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.258766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.258817 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.258894 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.259341 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54132: BlockingRecv error: recv got EOF from 127.0.0.1:54132 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260808 04:50:07.327028 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.325749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.22.2.189:34787)
0808 04:50:07.325922 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.325927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.326061 (+ 134us) server_negotiation.cc:407] Connection header received
0808 04:50:07.326259 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.326262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.326315 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.326405 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.326896 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54142: BlockingRecv error: recv got EOF from 127.0.0.1:54142 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 04:50:07.458654 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.457419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.22.2.189:34787)
0808 04:50:07.457608 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.457612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.457826 (+ 214us) server_negotiation.cc:407] Connection header received
0808 04:50:07.457947 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.457949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.458003 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.458087 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.458495 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54144: BlockingRecv error: recv got EOF from 127.0.0.1:54144 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 04:50:07.718125 23280 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:07.716963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54148 (local address 127.22.2.189:34787)
0808 04:50:07.717135 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:07.717140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:07.717263 (+ 123us) server_negotiation.cc:407] Connection header received
0808 04:50:07.717407 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:07.717409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:07.717460 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:07.717538 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:07.718009 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54148: BlockingRecv error: recv got EOF from 127.0.0.1:54148 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 04:50:07.986626 23422 catalog_manager.cc:1259] Loaded cluster ID: 0db7896f5ccd40a7947b584647915193
I20260808 04:50:07.986714 23422 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c: loading cluster ID for follower catalog manager: success
I20260808 04:50:07.988472 23422 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c: acquiring CA information for follower catalog manager: success
I20260808 04:50:07.989688 23422 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aa68c3a8aa7c42a095d5525c4bc4339c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 04:50:08.020372 23457 catalog_manager.cc:1259] Loaded cluster ID: 0db7896f5ccd40a7947b584647915193
I20260808 04:50:08.020447 23457 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df: loading cluster ID for follower catalog manager: success
I20260808 04:50:08.022133 23457 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df: acquiring CA information for follower catalog manager: success
I20260808 04:50:08.023295 23457 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a9d9eed33ad4483a4f07f03b1de49df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 04:50:08.236379 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:08.232283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.22.2.189:34787)
0808 04:50:08.232593 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:08.232600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:08.232621 (+ 21us) server_negotiation.cc:407] Connection header received
0808 04:50:08.232812 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:08.232816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:08.232959 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:08.233088 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:08.233366 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.233896 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.234800 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.235034 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.235147 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:08.235480 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:08.235940 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:08.236078 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:08.236144 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 04:50:08.236199 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":71,"threads_started":1}
I20260808 04:50:08.242903 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:08.239403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49728 (local address 127.22.2.188:38737)
0808 04:50:08.239662 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:08.239666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:08.239680 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:50:08.239829 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:08.239831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:08.239944 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:08.240040 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:08.240221 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.240769 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.241538 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.241833 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.241912 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:08.242156 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:08.242520 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:08.242608 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:08.242661 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 04:50:08.242703 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260808 04:50:08.250104 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:08.246003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54160 (local address 127.22.2.189:34787)
0808 04:50:08.246130 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:08.246136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:08.246282 (+ 146us) server_negotiation.cc:407] Connection header received
0808 04:50:08.246443 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:08.246446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:08.246500 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:08.246546 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:08.246974 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.247712 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.249060 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.249286 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.249400 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:08.249639 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:08.249713 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:08.249800 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:08.249891 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 04:50:08.249953 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 04:50:08.250254 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:08.246368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49738 (local address 127.22.2.188:38737)
0808 04:50:08.246486 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:08.246490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:08.246775 (+ 285us) server_negotiation.cc:407] Connection header received
0808 04:50:08.246897 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:08.246903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:08.246960 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:08.247045 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:08.247281 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.247805 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.249049 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.249357 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.249456 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:08.249792 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:08.249873 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:08.249938 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:08.250085 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 04:50:08.250141 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 04:50:08.250308 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:08.245683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.22.2.190:38865)
0808 04:50:08.245966 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:08.245971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:08.245986 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:50:08.246083 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:08.246087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:08.246257 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:08.246374 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:08.246743 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.247589 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.248737 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:08.248993 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:08.249097 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:08.249547 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:08.249869 (+ 322us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:08.249953 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:08.250083 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 04:50:08.250141 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":100,"threads_started":1}
I20260808 04:50:08.260159 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23060
I20260808 04:50:08.274672 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23180
I20260808 04:50:08.287026 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23287
2026-08-08T04:50:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3443 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:50:08 dist-test-slave-04bb krb5kdc[23604](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:50:08 dist-test-slave-04bb krb5kdc[23604](info): set up 2 sockets
Aug 08 04:50:08 dist-test-slave-04bb krb5kdc[23604](info): commencing operation
krb5kdc: starting...
W20260808 04:50:09.512846 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/test-user.keytab.
Aug 08 04:50:09 dist-test-slave-04bb krb5kdc[23604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:50:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 04:50:09.642968 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:43761
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:43761,127.22.2.189:37859,127.22.2.188:44237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:50:09.753827 23623 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 04:50:09.754083 23623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:09.754106 23623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:09.754143 23623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:50:09.758025 23623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:09.758073 23623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:09.758092 23623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:09.758121 23623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:09.758136 23623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:09.764729 23623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:43761,127.22.2.189:37859,127.22.2.188:44237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:43761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:09.765693 23623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:09.766649 23623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:09.775511 23632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:09.775532 23635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:09.775534 23630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:09.775770 23623 server_base.cc:1034] running on GCE node
I20260808 04:50:09.776456 23623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:09.777551 23623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:09.778728 23623 hybrid_clock.cc:648] HybridClock initialized: now 1786164609778702 us; error 41 us; skew 500 ppm
Aug 08 04:50:09 dist-test-slave-04bb krb5kdc[23604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164609, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:09.781766 23623 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 04:50:09.782696 23623 webserver.cc:460] Webserver started at http://127.22.2.190:37811/ using document root <none> and password file <none>
I20260808 04:50:09.783273 23623 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:09.783340 23623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:09.783599 23623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:09.785771 23623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/data/instance:
uuid: "4847d4244cf6462a8215c712933a307b"
format_stamp: "Formatted at 2026-08-08 04:50:09 on dist-test-slave-04bb"
server_key: "30ae9e0046d17a3770b1acf68bd88f02"
server_key_iv: "aa18f7900cc210f611a74b8101964c24"
server_key_version: "encryptionkey@0"
I20260808 04:50:09.786367 23623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/wal/instance:
uuid: "4847d4244cf6462a8215c712933a307b"
format_stamp: "Formatted at 2026-08-08 04:50:09 on dist-test-slave-04bb"
server_key: "30ae9e0046d17a3770b1acf68bd88f02"
server_key_iv: "aa18f7900cc210f611a74b8101964c24"
server_key_version: "encryptionkey@0"
I20260808 04:50:09.790323 23623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 04:50:09.793059 23644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:09.794250 23623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 04:50:09.794399 23623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/wal
uuid: "4847d4244cf6462a8215c712933a307b"
format_stamp: "Formatted at 2026-08-08 04:50:09 on dist-test-slave-04bb"
server_key: "30ae9e0046d17a3770b1acf68bd88f02"
server_key_iv: "aa18f7900cc210f611a74b8101964c24"
server_key_version: "encryptionkey@0"
I20260808 04:50:09.794509 23623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:09.815109 23623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:09.818059 23623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:09.818305 23623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:09.825371 23623 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:43761
I20260808 04:50:09.825675 23725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:43761 every 8 connection(s)
I20260808 04:50:09.826442 23623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/data/info.pb
I20260808 04:50:09.829025 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23623
I20260808 04:50:09.829172 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-0/wal/instance
I20260808 04:50:09.829569 22538 external_mini_cluster.cc:1428] Setting key 1a84b42a6cfb501d5a9b86dca1f2a528
I20260808 04:50:09.829604 23726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:09.834062 23726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.2.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.189@KRBTEST.COM" created.
Aug 08 04:50:09 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164609, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:50:09.850472 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:09.836476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43761 (local address 127.0.0.1:43166)
0808 04:50:09.836965 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:09.837015 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:09.837865 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:09.838741 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:09.838749 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:09.839059 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:09.839355 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:09.839368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:09.840850 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:09.840853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:09.841570 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:09.841579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:09.842332 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:09.842341 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:09.842572 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:09.843211 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:09.843239 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:09.844914 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:09.847487 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:09.847496 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:09.847504 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:09.847746 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:09.848151 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:09.848153 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:09.848154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:09.848269 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:09.849100 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:09.849105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:09.849231 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 04:50:09.849382 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 04:50:09.849670 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":69,"thread_start_us":123,"threads_started":1}
I20260808 04:50:09.850478 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:09.836478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43166 (local address 127.22.2.190:43761)
0808 04:50:09.837100 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:09.837114 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:09.837164 (+ 50us) server_negotiation.cc:407] Connection header received
0808 04:50:09.838196 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:09.838203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:09.838480 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:09.838731 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:09.839554 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:09.840713 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:09.841727 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:09.842133 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:09.842289 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:09.845068 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:09.845073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:09.845082 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:09.845103 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:09.847371 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:09.847878 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:09.847888 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:09.847893 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:09.848045 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:09.848370 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:09.848377 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:09.848380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:09.848975 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:09.849139 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:09.849581 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 04:50:09.849758 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":491,"thread_start_us":162,"threads_started":1}
I20260808 04:50:09.853103 23726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:09.853870 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:09.853562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54274)
0808 04:50:09.853700 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:09.853790 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 04:50:09.854100 23645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
I20260808 04:50:09.855044 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:09.854829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54278)
0808 04:50:09.854940 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:09.854988 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 04:50:09.855197 23726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:37859: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
I20260808 04:50:09.861945 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:37859
--webserver_interface=127.22.2.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:43761,127.22.2.189:37859,127.22.2.188:44237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:50:09.904449 23726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } attempt: 1
I20260808 04:50:09.905439 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:09.905127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54294)
0808 04:50:09.905288 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:09.905358 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 04:50:09.906186 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:09.905970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54296)
0808 04:50:09.906077 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:09.906128 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 04:50:09.906317 23726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:37859: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
I20260808 04:50:09.974473 23726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } attempt: 2
I20260808 04:50:09.975394 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:09.975077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54304)
0808 04:50:09.975248 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:09.975315 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 04:50:09.976125 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:09.975916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54316)
0808 04:50:09.976024 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:09.976073 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 04:50:09.976250 23726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:37859: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37859: connect: Connection refused (error 111)
W20260808 04:50:09.977994 23739 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 04:50:09.978291 23739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:09.978353 23739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:09.978421 23739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:50:09.982407 23739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:09.982481 23739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:09.982520 23739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:09.982553 23739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:09.982583 23739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:09.988396 23739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:43761,127.22.2.189:37859,127.22.2.188:44237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:37859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.2.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:09.989586 23739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:09.990659 23739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:09.999141 23747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:09.999152 23746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:09.999152 23750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:09.999423 23739 server_base.cc:1034] running on GCE node
I20260808 04:50:09.999992 23739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:10.001124 23739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:10.002321 23739 hybrid_clock.cc:648] HybridClock initialized: now 1786164610002291 us; error 48 us; skew 500 ppm
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:10.005136 23739 init.cc:377] Logged in from keytab as kudu/127.22.2.189@KRBTEST.COM (short username kudu)
I20260808 04:50:10.005937 23739 webserver.cc:460] Webserver started at http://127.22.2.189:32833/ using document root <none> and password file <none>
I20260808 04:50:10.006448 23739 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:10.006505 23739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:10.006661 23739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:10.008661 23739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/data/instance:
uuid: "4c7af3e3a52544919007e024e6a997d8"
format_stamp: "Formatted at 2026-08-08 04:50:10 on dist-test-slave-04bb"
server_key: "2a30a03a0673302ba21af885de174616"
server_key_iv: "09846f19264ff25f16a31edae6d4518f"
server_key_version: "encryptionkey@0"
I20260808 04:50:10.009174 23739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/wal/instance:
uuid: "4c7af3e3a52544919007e024e6a997d8"
format_stamp: "Formatted at 2026-08-08 04:50:10 on dist-test-slave-04bb"
server_key: "2a30a03a0673302ba21af885de174616"
server_key_iv: "09846f19264ff25f16a31edae6d4518f"
server_key_version: "encryptionkey@0"
I20260808 04:50:10.012866 23739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 04:50:10.015182 23760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:10.016392 23739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:50:10.016517 23739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/wal
uuid: "4c7af3e3a52544919007e024e6a997d8"
format_stamp: "Formatted at 2026-08-08 04:50:10 on dist-test-slave-04bb"
server_key: "2a30a03a0673302ba21af885de174616"
server_key_iv: "09846f19264ff25f16a31edae6d4518f"
server_key_version: "encryptionkey@0"
I20260808 04:50:10.016647 23739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:10.030390 23739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:10.033250 23739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:10.033461 23739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:10.040270 23739 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:37859
I20260808 04:50:10.040609 23835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:37859 every 8 connection(s)
I20260808 04:50:10.041270 23739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/data/info.pb
I20260808 04:50:10.044077 23836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:10.047886 23836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:10.049365 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23739
I20260808 04:50:10.049496 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-1/wal/instance
I20260808 04:50:10.049839 22538 external_mini_cluster.cc:1428] Setting key 001a8a102c591a018830d2aff43d6c3c
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.188@KRBTEST.COM" created.
I20260808 04:50:10.062829 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.049820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.22.2.190:43761)
0808 04:50:10.049969 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.049973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.051003 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 04:50:10.052059 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.052063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.052115 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.052181 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.053821 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.054363 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.055643 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.055898 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.056013 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.058511 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.058517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.058522 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.058556 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.060617 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.061108 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.061112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.061116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.061245 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.061621 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.061624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.061630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.061834 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.061916 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.062601 (+ 685us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.062702 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 04:50:10.063848 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.050329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43761 (local address 127.0.0.1:43168)
0808 04:50:10.050845 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.050889 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:10.051766 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:10.052403 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:10.052413 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:10.052979 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:10.053668 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.053685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.054497 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.054501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.055508 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.055519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.056055 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.056060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.056230 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.056884 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:10.056901 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:10.058330 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:10.060731 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.060735 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.060748 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.061001 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.061367 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.061374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.061376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.061522 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.061952 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:10.061962 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:10.062305 (+ 343us) client_negotiation.cc:769] Sending connection context
0808 04:50:10.062541 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 04:50:10.062914 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":144,"threads_started":1}
I20260808 04:50:10.065382 23836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:10.067416 23726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } attempt: 3
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164609, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 04:50:10.076578 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.065929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54322)
0808 04:50:10.066077 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.066088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:10.066146 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:10.066691 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:10.066694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:10.066872 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:10.066946 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.066951 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.068090 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.068096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.068858 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.068871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.069446 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.069449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.069539 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.070000 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:10.070012 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:10.072774 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:10.074882 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.074888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.074890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.075078 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.075384 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.075387 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.075389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.075493 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.076079 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:10.076082 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:10.076172 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 04:50:10.076259 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 04:50:10.076418 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 04:50:10.076833 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.066037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.22.2.189:37859)
0808 04:50:10.066371 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.066380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.066397 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:50:10.066430 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.066434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.066574 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.066714 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.067053 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.067965 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.068973 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.069331 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.069447 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.072915 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.072924 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.072935 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.072955 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.074768 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.075185 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.075191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.075193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.075285 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.075606 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.075610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.075611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.075950 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.076154 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.076385 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.076613 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":139,"threads_started":1}
I20260808 04:50:10.078490 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.067987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54330)
0808 04:50:10.068167 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.068180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:10.068247 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:10.068990 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:10.068996 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:10.069184 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:10.069335 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.069346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.070428 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.070433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.070972 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.070981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.071558 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.071562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.071694 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.072189 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:10.072204 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:10.073997 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:10.076837 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.076844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.076849 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.077112 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.077431 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.077434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.077436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.077522 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.077964 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:10.077966 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:10.078050 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 04:50:10.078149 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 04:50:10.078364 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260808 04:50:10.078517 23846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.068058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54330 (local address 127.22.2.189:37859)
0808 04:50:10.068735 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.068743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.068756 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:50:10.068790 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.068793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.068844 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.068938 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.069518 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.070313 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.071089 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.071437 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.071544 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.074134 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.074137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.074139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.074160 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.076668 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.077263 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.077265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.077266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.077337 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.077616 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.077619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.077621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.077842 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.077943 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.078247 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.078354 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":116,"threads_started":1}
I20260808 04:50:10.079064 23836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
I20260808 04:50:10.079365 23726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:10.080060 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.079786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41848)
0808 04:50:10.079950 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.079996 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 04:50:10.080060 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.079762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41840)
0808 04:50:10.079916 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.079996 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 04:50:10.080365 23761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
I20260808 04:50:10.080466 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
/tmp/dist-test-taskzqcaaz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:44237
--webserver_interface=127.22.2.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:43761,127.22.2.189:37859,127.22.2.188:44237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:50:10.080845 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.080553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41864)
0808 04:50:10.080739 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.080783 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260808 04:50:10.080986 23726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:44237: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
I20260808 04:50:10.081516 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.081292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41868)
0808 04:50:10.081420 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.081470 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 04:50:10.081682 23836 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:44237: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
I20260808 04:50:10.112145 23726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } attempt: 1
I20260808 04:50:10.113067 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.112704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41872)
0808 04:50:10.112885 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.112964 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 04:50:10.113855 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.113594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41874)
0808 04:50:10.113726 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.113782 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 04:50:10.114012 23726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:44237: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
I20260808 04:50:10.130903 23836 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } attempt: 1
I20260808 04:50:10.131754 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.131422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41890)
0808 04:50:10.131588 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.131662 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 04:50:10.132515 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.132285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41906)
0808 04:50:10.132396 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.132450 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 04:50:10.132671 23836 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:44237: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
I20260808 04:50:10.189172 23726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } attempt: 2
I20260808 04:50:10.190070 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.189721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41912)
0808 04:50:10.189898 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.189971 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 04:50:10.190843 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.190602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41922)
0808 04:50:10.190715 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.190769 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 04:50:10.190989 23726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:44237: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
W20260808 04:50:10.199039 23849 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 04:50:10.199304 23849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:50:10.199335 23849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:50:10.199374 23849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 04:50:10.200817 23836 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } attempt: 2
I20260808 04:50:10.201571 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.201277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41934)
0808 04:50:10.201424 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.201487 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 04:50:10.202303 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.202075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41936)
0808 04:50:10.202184 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.202234 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 04:50:10.202457 23836 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:44237: Network error: Client connection negotiation failed: client connection to 127.22.2.188:44237: connect: Connection refused (error 111)
W20260808 04:50:10.203898 23849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:50:10.203946 23849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:50:10.203986 23849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:50:10.204026 23849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:50:10.204059 23849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:50:10.209403 23849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:43761,127.22.2.189:37859,127.22.2.188:44237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:44237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.2.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 04:05:18 UTC on ccycloud.quasar-drfdgt.root.comops.site
build id 4574101
I20260808 04:50:10.210467 23849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:50:10.211498 23849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:50:10.219995 23856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:10.220005 23859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:50:10.220005 23854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:50:10.220358 23849 server_base.cc:1034] running on GCE node
I20260808 04:50:10.220990 23849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:50:10.222061 23849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:50:10.223245 23849 hybrid_clock.cc:648] HybridClock initialized: now 1786164610223236 us; error 41 us; skew 500 ppm
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:50:10.226035 23849 init.cc:377] Logged in from keytab as kudu/127.22.2.188@KRBTEST.COM (short username kudu)
I20260808 04:50:10.226966 23849 webserver.cc:460] Webserver started at http://127.22.2.188:37141/ using document root <none> and password file <none>
I20260808 04:50:10.227543 23849 fs_manager.cc:359] Metadata directory not provided
I20260808 04:50:10.227633 23849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:50:10.227874 23849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:50:10.229970 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/data/instance:
uuid: "1dee8bed78d44273907f9871cc8d1fc5"
format_stamp: "Formatted at 2026-08-08 04:50:10 on dist-test-slave-04bb"
server_key: "241b803e23ce2992c6ab865e431bde60"
server_key_iv: "8e6e90c7b8b3a7547d7f061dc1e4218e"
server_key_version: "encryptionkey@0"
I20260808 04:50:10.230545 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/wal/instance:
uuid: "1dee8bed78d44273907f9871cc8d1fc5"
format_stamp: "Formatted at 2026-08-08 04:50:10 on dist-test-slave-04bb"
server_key: "241b803e23ce2992c6ab865e431bde60"
server_key_iv: "8e6e90c7b8b3a7547d7f061dc1e4218e"
server_key_version: "encryptionkey@0"
I20260808 04:50:10.234254 23849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 04:50:10.236685 23867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:50:10.237854 23849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 04:50:10.237993 23849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/wal
uuid: "1dee8bed78d44273907f9871cc8d1fc5"
format_stamp: "Formatted at 2026-08-08 04:50:10 on dist-test-slave-04bb"
server_key: "241b803e23ce2992c6ab865e431bde60"
server_key_iv: "8e6e90c7b8b3a7547d7f061dc1e4218e"
server_key_version: "encryptionkey@0"
I20260808 04:50:10.238101 23849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:50:10.263197 23849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:50:10.265990 23849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:50:10.266197 23849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:50:10.273252 23849 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:44237
I20260808 04:50:10.273527 23938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:44237 every 8 connection(s)
I20260808 04:50:10.274212 23849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/data/info.pb
I20260808 04:50:10.277066 23939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:50:10.277383 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu as pid 23849
I20260808 04:50:10.277516 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzqcaaz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786164310373382-22538-0/minicluster-data/master-2/wal/instance
I20260808 04:50:10.277874 22538 external_mini_cluster.cc:1428] Setting key 0e31aa1409e403b8ec81ac746931f44a
I20260808 04:50:10.282001 23939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:50:10.290140 23726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } attempt: 3
I20260808 04:50:10.293248 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.279177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41950 (local address 127.22.2.188:44237)
0808 04:50:10.279569 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.279586 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.279622 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:50:10.280291 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.280316 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.280657 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.280926 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.281859 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.282885 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.283957 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.284721 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.284893 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.287790 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.287805 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.287815 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.287847 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.289569 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.290072 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.290077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.290081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.290232 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.290556 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.290559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.290561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.290976 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.291471 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.291959 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.292238 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
I20260808 04:50:10.293593 23836 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } attempt: 3
I20260808 04:50:10.294100 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.283758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43174 (local address 127.22.2.190:43761)
0808 04:50:10.283886 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.283891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.284135 (+ 244us) server_negotiation.cc:407] Connection header received
0808 04:50:10.284262 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.284265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.284322 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.284391 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.285328 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.286128 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.287369 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.287576 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.287659 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.290233 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.290236 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.290238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.290286 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.292093 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.292693 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.292697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.292699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.292787 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.293084 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.293087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.293089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.293289 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.293388 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.293844 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.293959 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 04:50:10.294935 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.283608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43761 (local address 127.0.0.1:43174)
0808 04:50:10.284009 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.284054 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:10.284167 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:10.284464 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:10.284470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:10.284945 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:10.285179 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.285188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.286261 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.286264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.287192 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.287207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.287740 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.287747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.287844 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.288483 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:10.288494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:10.290087 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:10.292219 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.292228 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.292234 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.292588 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.292871 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.292874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.292875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.292972 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.293445 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:10.293451 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:10.293589 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 04:50:10.294279 (+ 690us) client_negotiation.cc:240] Negotiation successful
0808 04:50:10.294468 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":24,"thread_start_us":103,"threads_started":1}
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164609, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:50:10.296777 23939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:50:10.300341 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.290953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41956)
0808 04:50:10.291089 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.291105 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:10.291183 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:10.291517 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:10.291521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:10.291698 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:10.291802 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.291810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.292803 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.292808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.293547 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.293557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.294084 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.294088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.294196 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.294679 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:10.294693 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:10.296448 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:10.298695 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.298699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.298701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.298968 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.299285 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.299289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.299291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.299387 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.299813 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:10.299817 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:10.299934 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 04:50:10.300037 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 04:50:10.300177 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 04:50:10.300885 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.290960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41956 (local address 127.22.2.188:44237)
0808 04:50:10.291276 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.291286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.291301 (+ 15us) server_negotiation.cc:407] Connection header received
0808 04:50:10.291349 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.291352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.291397 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.291499 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.291946 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.292685 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.293653 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.293950 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.294064 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.296639 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.296642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.296645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.296668 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.298585 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.299061 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.299065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.299068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.299177 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.299482 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.299487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.299489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.299686 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.299797 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.300567 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.300673 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":93,"threads_started":1}
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 04:50:10.304206 23726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b: Bootstrap starting.
I20260808 04:50:10.305244 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.294756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41960)
0808 04:50:10.295074 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.295089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:10.295170 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:10.295648 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:10.295651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:10.295850 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:10.295991 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.296000 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.296950 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.296954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.297840 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.297853 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.298457 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.298464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.298576 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.299061 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:10.299077 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:10.300649 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:10.303199 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.303202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.303204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.303405 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.303717 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.303720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.303722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.303810 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.304474 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:10.304477 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:10.304601 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 04:50:10.304708 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 04:50:10.305089 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260808 04:50:10.305886 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.295324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.22.2.188:44237)
0808 04:50:10.295426 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.295430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.295444 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:50:10.295476 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.295479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.295517 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.295610 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.296116 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.296812 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.298001 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.298338 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.298437 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.301390 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.301394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.301396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.301422 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.303081 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.303538 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.303541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.303543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.303622 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.303902 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.303906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.303907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.304116 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.304228 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.305557 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.305745 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 04:50:10.307142 23726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:10.307883 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.296622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.22.2.190:43761)
0808 04:50:10.296716 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.296720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.296875 (+ 155us) server_negotiation.cc:407] Connection header received
0808 04:50:10.297049 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.297052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.297093 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.297139 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.298018 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.298695 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.299888 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.300241 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.300351 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.303984 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.303988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.303991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.304014 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.305608 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.306416 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.306420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.306422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.306527 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.306861 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.306864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.306866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.307076 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.307247 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.307617 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.307724 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 04:50:10.307905 23726 log.cc:824] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:50:10.309190 23836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8: Bootstrap starting.
I20260808 04:50:10.310312 23726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b: No bootstrap required, opened a new log
I20260808 04:50:10.311905 23836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:10.312774 23836 log.cc:824] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8: Log is configured to *not* fsync() on all Append() calls
I20260808 04:50:10.313380 23726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } }
I20260808 04:50:10.313630 23726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4847d4244cf6462a8215c712933a307b, State: Initialized, Role: FOLLOWER
I20260808 04:50:10.313987 23726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } }
I20260808 04:50:10.314476 23950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.314689 23950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:10.314903 23836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8: No bootstrap required, opened a new log
I20260808 04:50:10.315344 23726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 04:50:10.319453 23836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } }
W20260808 04:50:10.319878 23652 tablet.cc:2369] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:50:10.320077 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.309815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54348 (local address 127.22.2.189:37859)
0808 04:50:10.309933 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.309937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.310132 (+ 195us) server_negotiation.cc:407] Connection header received
0808 04:50:10.310172 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.310175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.310215 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.310285 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.311125 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.311854 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.312978 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.313337 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.313455 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.316220 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.316224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.316227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.316257 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.318010 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.318521 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.318524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.318526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.318615 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.318934 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.318938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.318940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.319175 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.319367 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.319793 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.319938 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 04:50:10.320586 23969 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:50:10.320641 23969 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:50:10.321256 23836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7af3e3a52544919007e024e6a997d8, State: Initialized, Role: FOLLOWER
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 04:50:10.321740 23836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } }
I20260808 04:50:10.322772 23972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.322925 23972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:10.323658 23836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:50:10.320442 23623 master_runner.cc:186] Error getting master registration for 127.22.2.189:37859: OK, instance_id { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" instance_seqno: 1786164610038330 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 04:50:10.324326 23623 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 04:50:10.325141 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:50:10.326081 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.297280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37859 (local address 127.0.0.1:54346)
0808 04:50:10.299964 (+ 2684us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.299978 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:10.300090 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:10.300444 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:10.300447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:10.300634 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:10.300715 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.300721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.304618 (+ 3897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.304622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.305504 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.305519 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.306517 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.306520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.306622 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.317082 (+ 10460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:10.317114 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:10.321958 (+ 4844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:10.324256 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.324259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.324261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.324584 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.324927 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.324932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.324934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.325028 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.325486 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:10.325489 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:10.325588 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 04:50:10.325677 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 04:50:10.325835 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2614}
I20260808 04:50:10.326474 23688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4847d4244cf6462a8215c712933a307b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43178
I20260808 04:50:10.326586 23846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.297586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54346 (local address 127.22.2.189:37859)
0808 04:50:10.297729 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.297735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.300032 (+ 2297us) server_negotiation.cc:407] Connection header received
0808 04:50:10.300195 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.300198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.300243 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.300291 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.300851 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.301509 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.305684 (+ 4175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.306093 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.306186 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.322123 (+ 15937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.322126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.322129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.322155 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.324130 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.324717 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.324720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.324722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.324820 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.325139 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.325142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.325144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.325348 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.325439 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.326314 (+ 875us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.326451 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 04:50:10.326720 23688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 04:50:10.326855 23688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:10.327596 23939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:50:10.328269 23688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } }
I20260808 04:50:10.329250 23688 leader_election.cc:288] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [CANDIDATE]: Term 1 election: Requested vote from peers 4c7af3e3a52544919007e024e6a997d8 (127.22.2.189:37859), 1dee8bed78d44273907f9871cc8d1fc5 (127.22.2.188:44237)
I20260808 04:50:10.329891 23799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4847d4244cf6462a8215c712933a307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4c7af3e3a52544919007e024e6a997d8"
I20260808 04:50:10.330127 23799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:10.331462 23799 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4847d4244cf6462a8215c712933a307b in term 1.
W20260808 04:50:10.331755 23989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:50:10.331949 23989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:50:10.332293 23645 leader_election.cc:302] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4847d4244cf6462a8215c712933a307b, 4c7af3e3a52544919007e024e6a997d8; no voters:
I20260808 04:50:10.332626 23950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:50:10.332841 23950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [term 1 LEADER]: Becoming Leader. State: Replica: 4847d4244cf6462a8215c712933a307b, State: Running, Role: LEADER
Aug 08 04:50:10 dist-test-slave-04bb krb5kdc[23604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786164610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 04:50:10.333144 23950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } }
I20260808 04:50:10.337203 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.328085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:44237 (local address 127.0.0.1:41972)
0808 04:50:10.328306 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 04:50:10.328333 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 04:50:10.328426 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:50:10.328773 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:50:10.328776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:50:10.328953 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:50:10.329072 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.329079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.330032 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.330035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.330781 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:50:10.330792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.331465 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.331468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:50:10.331551 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.332072 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:50:10.332088 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:50:10.333551 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:50:10.335551 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.335554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.335556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.335794 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.336137 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:50:10.336140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:50:10.336142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:50:10.336239 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:50:10.336685 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:50:10.336688 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:50:10.336782 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 04:50:10.336903 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 04:50:10.337037 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 04:50:10.337738 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.328160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.22.2.188:44237)
0808 04:50:10.328268 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.328272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.328404 (+ 132us) server_negotiation.cc:407] Connection header received
0808 04:50:10.328551 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.328555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.328603 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.328677 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.329190 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.329906 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.330919 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.331231 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.331347 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.333706 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.333709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.333712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.333740 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.335431 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.335917 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.335921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.335935 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.336030 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.336345 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.336350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.336353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.336586 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.336695 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.337491 (+ 796us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.337597 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 04:50:10.338140 24000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4847d4244cf6462a8215c712933a307b. Latest consensus state: current_term: 1 leader_uuid: "4847d4244cf6462a8215c712933a307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.338322 24000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:10.341246 23939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5: Bootstrap starting.
I20260808 04:50:10.344035 23939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5: Neither blocks nor log segments found. Creating new log.
I20260808 04:50:10.344942 23739 master_runner.cc:186] Error getting master registration for 127.22.2.188:44237: OK, instance_id { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" instance_seqno: 1786164610271274 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 04:50:10.348376 23739 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 04:50:10.348779 24004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:50:10.349282 23939 log.cc:824] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5: Log is configured to *not* fsync() on all Append() calls
I20260808 04:50:10.349951 24004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:50:10.352042 23939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5: No bootstrap required, opened a new log
I20260808 04:50:10.356247 23939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } }
I20260808 04:50:10.356505 23939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dee8bed78d44273907f9871cc8d1fc5, State: Initialized, Role: FOLLOWER
I20260808 04:50:10.357357 23939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } }
I20260808 04:50:10.359724 23904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4847d4244cf6462a8215c712933a307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1dee8bed78d44273907f9871cc8d1fc5"
I20260808 04:50:10.359972 23904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:50:10.361330 23904 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4847d4244cf6462a8215c712933a307b in term 1.
I20260808 04:50:10.359655 23939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:50:10.362142 23846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.340882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.22.2.189:37859)
0808 04:50:10.340999 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.341003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.344850 (+ 3847us) server_negotiation.cc:407] Connection header received
0808 04:50:10.345045 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.345048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.345096 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.345149 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.347478 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.348269 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.351431 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.351767 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.351865 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.355752 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.355756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.355759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.355786 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.357698 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.360745 (+ 3047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.360749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.360751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.360849 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.361140 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.361144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.361146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.361351 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.361481 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.361848 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.361986 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 04:50:10.364876 24012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.365058 24012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:10.370286 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.341264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.22.2.188:44237)
0808 04:50:10.341374 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.341378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.346286 (+ 4908us) server_negotiation.cc:407] Connection header received
0808 04:50:10.346449 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.346452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.346494 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.346541 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.349547 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.350282 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.352564 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.352899 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.352972 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.358393 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.358397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.358400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.358425 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.360339 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.368592 (+ 8253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.368596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.368598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.368725 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.369072 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.369077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.369078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.369301 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.369417 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.370024 (+ 607us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.370134 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 04:50:10.370457 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:10.341744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.22.2.190:43761)
0808 04:50:10.341864 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.341869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.352299 (+ 10430us) server_negotiation.cc:407] Connection header received
0808 04:50:10.352493 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.352496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.352564 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.352617 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.353561 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.354295 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.355435 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:10.355764 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:10.355859 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:10.357777 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:50:10.357781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:50:10.357784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:50:10.357826 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:50:10.359518 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.368962 (+ 9444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.368966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.368968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.369068 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:50:10.369412 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:50:10.369416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:50:10.369418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:50:10.369649 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:50:10.369764 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:10.370150 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 04:50:10.370276 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 04:50:10.368605 24029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:50:10.371246 24029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:50:10.374625 23904 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [term 1 FOLLOWER]: Refusing update from remote peer 4847d4244cf6462a8215c712933a307b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:50:10.374704 23799 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [term 1 FOLLOWER]: Refusing update from remote peer 4847d4244cf6462a8215c712933a307b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:50:10.375361 23950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:50:10.375799 23950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:50:10.384809 24012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4847d4244cf6462a8215c712933a307b. Latest consensus state: current_term: 1 leader_uuid: "4847d4244cf6462a8215c712933a307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.384956 24012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:10.387934 23972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4847d4244cf6462a8215c712933a307b. Latest consensus state: current_term: 1 leader_uuid: "4847d4244cf6462a8215c712933a307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.388059 23972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:10.395898 24000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4847d4244cf6462a8215c712933a307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.396030 24000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:10.396153 24011 mvcc.cc:205] Tried to move back new op lower bound from 7316130244088152064 to 7316130243928100864. Current Snapshot: MvccSnapshot[applied={T|T < 7316130244088152064}]
I20260808 04:50:10.397397 24012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4847d4244cf6462a8215c712933a307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.397508 24012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:10.399708 23950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4847d4244cf6462a8215c712933a307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.399844 23950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b [sys.catalog]: This master's current role is: LEADER
I20260808 04:50:10.400795 23972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4847d4244cf6462a8215c712933a307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4847d4244cf6462a8215c712933a307b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43761 } } peers { permanent_uuid: "4c7af3e3a52544919007e024e6a997d8" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37859 } } peers { permanent_uuid: "1dee8bed78d44273907f9871cc8d1fc5" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 44237 } } }
I20260808 04:50:10.400921 23972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:50:10.404626 24031 mvcc.cc:205] Tried to move back new op lower bound from 7316130244088152064 to 7316130243928100864. Current Snapshot: MvccSnapshot[applied={T|T < 7316130244088152064}]
I20260808 04:50:10.405063 24034 mvcc.cc:205] Tried to move back new op lower bound from 7316130244088152064 to 7316130243928100864. Current Snapshot: MvccSnapshot[applied={T|T < 7316130244088152064}]
I20260808 04:50:10.410615 24004 catalog_manager.cc:1347] Generated new cluster ID: d6d6ec494c2e485fa64a32b405d04e57
I20260808 04:50:10.410691 24004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:50:10.464090 24004 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:50:10.465551 24004 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:50:10.487644 24004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4847d4244cf6462a8215c712933a307b: Generated new TSK 0
I20260808 04:50:10.488511 24004 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 04:50:10.526103 23736 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.522758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.22.2.190:43761)
0808 04:50:10.522900 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.522905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.524828 (+ 1923us) server_negotiation.cc:407] Connection header received
0808 04:50:10.525119 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.525123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.525180 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.525243 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.525953 (+ 710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43198: BlockingRecv error: recv got EOF from 127.0.0.1:43198 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 04:50:10.527549 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.523682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.22.2.189:37859)
0808 04:50:10.523803 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.523809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.526613 (+ 2804us) server_negotiation.cc:407] Connection header received
0808 04:50:10.526786 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.526790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.526857 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.526912 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.527414 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54372: BlockingRecv error: recv got EOF from 127.0.0.1:54372 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 04:50:10.529009 23942 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.524442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.22.2.188:44237)
0808 04:50:10.524584 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.524589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.528073 (+ 3484us) server_negotiation.cc:407] Connection header received
0808 04:50:10.528184 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.528187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.528239 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.528331 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.528892 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41996: BlockingRecv error: recv got EOF from 127.0.0.1:41996 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 04:50:10.533799 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.531842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.22.2.189:37859)
0808 04:50:10.531947 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.531951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.532662 (+ 711us) server_negotiation.cc:407] Connection header received
0808 04:50:10.532931 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.532935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.533022 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.533104 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.533691 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54376: BlockingRecv error: recv got EOF from 127.0.0.1:54376 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 04:50:10.539525 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.537816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.22.2.189:37859)
0808 04:50:10.537926 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.537931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.538492 (+ 561us) server_negotiation.cc:407] Connection header received
0808 04:50:10.538706 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.538710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.538775 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.538836 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.539417 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54388: BlockingRecv error: recv got EOF from 127.0.0.1:54388 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 04:50:10.545686 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.544001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.22.2.189:37859)
0808 04:50:10.544128 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.544133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.544690 (+ 557us) server_negotiation.cc:407] Connection header received
0808 04:50:10.544917 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.544921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.544976 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.545034 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.545583 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54390: BlockingRecv error: recv got EOF from 127.0.0.1:54390 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 04:50:10.553857 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.552119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54406 (local address 127.22.2.189:37859)
0808 04:50:10.552265 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.552269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.552829 (+ 560us) server_negotiation.cc:407] Connection header received
0808 04:50:10.553048 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.553052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.553125 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.553188 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.553755 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54406: BlockingRecv error: recv got EOF from 127.0.0.1:54406 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 04:50:10.565717 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.564353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.22.2.189:37859)
0808 04:50:10.564467 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.564471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.564804 (+ 333us) server_negotiation.cc:407] Connection header received
0808 04:50:10.564935 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.564938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.564998 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.565102 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.565601 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54414: BlockingRecv error: recv got EOF from 127.0.0.1:54414 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 04:50:10.584496 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.583446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.22.2.189:37859)
0808 04:50:10.583603 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.583607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.583756 (+ 149us) server_negotiation.cc:407] Connection header received
0808 04:50:10.583867 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.583869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.583915 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.583984 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.584370 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54420: BlockingRecv error: recv got EOF from 127.0.0.1:54420 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 04:50:10.619858 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.618581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54436 (local address 127.22.2.189:37859)
0808 04:50:10.618715 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.618719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.618903 (+ 184us) server_negotiation.cc:407] Connection header received
0808 04:50:10.619026 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.619029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.619100 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.619200 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.619731 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54436: BlockingRecv error: recv got EOF from 127.0.0.1:54436 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 04:50:10.687314 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.686166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.22.2.189:37859)
0808 04:50:10.686342 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.686346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.686417 (+ 71us) server_negotiation.cc:407] Connection header received
0808 04:50:10.686549 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.686551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.686602 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.686675 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.687135 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54442: BlockingRecv error: recv got EOF from 127.0.0.1:54442 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 04:50:10.819281 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:10.817669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54454 (local address 127.22.2.189:37859)
0808 04:50:10.817839 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:10.817844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:10.817965 (+ 121us) server_negotiation.cc:407] Connection header received
0808 04:50:10.818154 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:10.818157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:10.818214 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:10.818637 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:10.819125 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54454: BlockingRecv error: recv got EOF from 127.0.0.1:54454 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 04:50:11.078877 23846 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:50:11.077789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54464 (local address 127.22.2.189:37859)
0808 04:50:11.077961 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:11.077966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:11.077982 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:50:11.078133 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:11.078136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:11.078192 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:11.078269 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:50:11.078754 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54464: BlockingRecv error: recv got EOF from 127.0.0.1:54464 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 04:50:11.333659 23989 catalog_manager.cc:1259] Loaded cluster ID: d6d6ec494c2e485fa64a32b405d04e57
I20260808 04:50:11.333778 23989 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8: loading cluster ID for follower catalog manager: success
I20260808 04:50:11.335494 23989 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8: acquiring CA information for follower catalog manager: success
I20260808 04:50:11.336733 23989 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4c7af3e3a52544919007e024e6a997d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 04:50:11.372684 24029 catalog_manager.cc:1259] Loaded cluster ID: d6d6ec494c2e485fa64a32b405d04e57
I20260808 04:50:11.372769 24029 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5: loading cluster ID for follower catalog manager: success
I20260808 04:50:11.374421 24029 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5: acquiring CA information for follower catalog manager: success
I20260808 04:50:11.375581 24029 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1dee8bed78d44273907f9871cc8d1fc5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 04:50:11.597307 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:11.593293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.22.2.189:37859)
0808 04:50:11.593591 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:11.593595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:11.593612 (+ 17us) server_negotiation.cc:407] Connection header received
0808 04:50:11.593736 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:11.593739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:11.593876 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:11.593980 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:11.594269 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.594862 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.595725 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.595965 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.596046 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:11.596334 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:11.596880 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:11.596981 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:11.597110 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 04:50:11.597165 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":92,"threads_started":1}
I20260808 04:50:11.604233 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:11.600904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.22.2.188:44237)
0808 04:50:11.601184 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:11.601188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:11.601202 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:50:11.601304 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:11.601307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:11.601422 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:11.601516 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:11.601683 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.602165 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.602930 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.603138 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.603215 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:11.603471 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:11.603853 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:11.603940 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:11.603997 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 04:50:11.604040 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":124,"threads_started":1}
I20260808 04:50:11.611610 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:11.607944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.22.2.188:44237)
0808 04:50:11.608048 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:11.608053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:11.608244 (+ 191us) server_negotiation.cc:407] Connection header received
0808 04:50:11.608404 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:11.608409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:11.608460 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:11.608572 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:11.608778 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.609470 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.610413 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.610693 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.610791 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:11.611078 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:11.611149 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:11.611220 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:11.611399 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 04:50:11.611476 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 04:50:11.611761 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:11.607205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.22.2.189:37859)
0808 04:50:11.607386 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:11.607389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:11.607808 (+ 419us) server_negotiation.cc:407] Connection header received
0808 04:50:11.608016 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:11.608019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:11.608076 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:11.608126 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:11.608470 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.609282 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.610502 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.610836 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.610933 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:11.611256 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:11.611358 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:11.611428 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:11.611570 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 04:50:11.611633 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260808 04:50:11.611965 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:50:11.607157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.22.2.190:43761)
0808 04:50:11.607452 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 04:50:11.607459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:50:11.607475 (+ 16us) server_negotiation.cc:407] Connection header received
0808 04:50:11.607594 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:50:11.607597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:50:11.607764 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:50:11.608164 (+ 400us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:50:11.608294 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.609030 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.610215 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:50:11.610569 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:50:11.610667 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:50:11.610981 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:50:11.611304 (+ 323us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:50:11.611708 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0808 04:50:11.611743 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 04:50:11.611817 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":93,"threads_started":1}
I20260808 04:50:11.614912 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23623
I20260808 04:50:11.628242 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23739
I20260808 04:50:11.640661 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzqcaaz/build/debug/bin/kudu with pid 23849
2026-08-08T04:50:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3354 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6798 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (301262 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST